Here's the revised LinkedIn post with points 7 and 8 integrated into points 2 and 3: 🎄✨ **Boost Your Holiday Spirit with Azure AI! 🎄✨ As we gear up for the holiday season, what better way to bring innovation to your business than by using cutting-edge Azure AI technologies? From personalized customer experiences to festive-themed data insights, here’s how Azure AI can help elevate your holiday initiatives: 🎅 1. Azure OpenAI Service for Creative Content Kickstart the holiday cheer by using Azure OpenAI to create engaging holiday content. From personalized greeting messages to festive social media posts, the GPT models can assist you in generating creative text in a snap. 🎨 Step-by-step: Use GPT to draft festive email newsletters, promotions, or customer-facing messages. Train models on your specific brand voice for customized holiday greetings. 🎁 2. Azure AI Services for Image Recognition and Generation Enhance your holiday product offerings by leveraging image recognition to identify and categorize holiday-themed products. Additionally, create stunning holiday-themed visuals with DALL-E. Generate unique images from text descriptions to make your holiday marketing materials stand out. 📸 Step-by-step: Use Azure Computer Vision to analyze product images and automatically categorize seasonal items. Implement the AI model in e-commerce platforms to help customers find holiday-specific products faster. Use DALL-E to generate holiday-themed images based on your descriptions. Customize and refine the images to fit your brand’s style. Incorporate these visuals into your marketing campaigns. ✨ 3. Azure AI Speech Services for Holiday Customer Interaction and Audio Generation Transform your customer service experience with Azure’s Speech-to-Text and Text-to-Speech services. You can create festive voice assistants or add holiday-themed voices to your customer support lines for a warm, personalized experience. Additionally, add a festive touch to your audio content with Azure OpenAI. Use models like Whisper for high-quality speech-to-text and text-to-speech conversions, perfect for creating holiday-themed audio messages and voice assistants. 🎙️ Step-by-step: Use Speech-to-Text to transcribe customer feedback or support requests in real-time. Build a holiday-themed voice model using Text-to-Speech for interactive voice assistants. Use Whisper to transcribe holiday messages or convert text to festive audio. Customize the audio to match your brand’s tone and style. Implement these audio clips in customer interactions or marketing materials. 🎄 4. Azure Machine Learning for Predictive Holiday Trends Stay ahead of holiday trends with Azure ML models. Use AI to analyze customer behavior, forecast demand for holiday products, and manage stock levels efficiently. Predict what your customers need before they even ask! 📊 Step-by-step: Use Azure ML to train models on historical sales data to predict trends in holiday shopping. Build dashboards using Power BI integrated with Azure for real-time tracking of holiday performance metrics. 🔔 5. Azure AI for Sentiment Analysis Understand the holiday mood of your customers by implementing sentiment analysis on social media, reviews, and feedback. Gauge the public sentiment around your brand during the festive season and respond accordingly. 📈 Step-by-step: Use Text Analytics for sentiment analysis on customer feedback, reviews, or social media posts. Generate insights and adapt your holiday marketing based on customer sentiment trends. 🌟 6. Latest Azure AI Open Models Explore the newest Azure AI models to bring even more innovation to your holiday projects: GPT-4o and GPT-4 Turbo: These models offer enhanced capabilities for understanding and generating natural language and code, perfect for creating sophisticated holiday content. Embeddings: Use these models to convert holiday-related text into numerical vectors for improved text similarity and search capabilities. 🔧 7. Azure AI Foundry Leverage Azure AI Foundry to build, deploy, and scale AI-driven applications. This platform provides everything you need to customize, host, run, and manage AI applications, ensuring your holiday projects are innovative and efficient 🎉 Conclusion: With Azure AI, the possibilities to brighten your business this holiday season are endless! Whether it's automating your operations or delivering personalized customer experiences, Azure's AI models can help you stay ahead of the game and spread holiday joy. Wishing everyone a season filled with innovation and success! 🎄✨

The o1 model is coming soon to the Microsoft OpenAI Service. This model brings advanced capabilities and improvements that will enable developers to apply reasoning capabilities to tasks such as inventory management, customer support inquiries, financial analysis and more.   Announcing the o1 model in Azure OpenAI Service   Also, as we continue to push the boundaries of AI capabilities, we are thrilled to announce several new fine-tuning features in Azure OpenAI Service. Learn more about o-1 mini-reinforcement fine tuning (optimize model behavior in highly complex or dynamic environments), direct preference optimization (adjust model weights based on human preferences), prompt caching (reduce request latency and costs by reusing recently seen input tokens) and more! Introducing New Fine-tuning Techniques and Capabilities in Azure OpenAI Service

We’re excited to provide an update on timing and billing clarifications for the Microsoft Entra ID Governance for guests add-on, following up on our previous blog post about Microsoft Entra ID Governance licensing clarifications. Timing update  We’re pleased to announce that the general availability for the ID Governance for guests add-on is now scheduled for the second quarter of 2025. This timeline allows us to ensure a smooth and seamless rollout for all our customers.  Billing clarity  We previously have highlighted that ID Governance for guests will be billed at $0.75 per monthly governed identity. To provide additional clarity on billing, we want to highlight a few key points:  Only features specific to Microsoft Entra ID Governance are eligible to be billed through the ID Governance for guests add-on. Governance features included in P2 will not be billed. Visit Microsoft Entra ID Governance licensing fundamentals - Microsoft Entra ID Governance | Microsoft Learn for a list of features by product. Only users with a UserType of Guest are in scope to be billed. Billing is consumptive and is charged based on usage every month. A single guest user identity that has a governance action applied in any month will only incur one $0.75 charge each month, regardless of the number of governance actions performed for that user in that month. Examples  Scenario 1: Automating access package assignments  March:  Contoso creates an auto-assignment policy that assigns an access package to 500 guest users. Contoso IT runs the guest conversion API for another set of 500 guest users. Billing: For March, Contoso is billed $750. This includes $375 for the auto-assignment policy (500 users x $0.75) and $375 for the guest conversion API (500 users x $0.75).   April:  The 500 guest users who were auto-assigned an access package in March retain their assignments but are not billed since there was not an explicit action taken on these users in April. Additionally, Contoso IT runs an inactive access review (AR) on 100 guests. Billing: For April, Contoso is billed $75 for the inactive AR (100 users x $0.75).   Scenario 2: Lifecycle workflow and access review for inactive users  March:  Fabrikam executes a lifecycle workflow for 300 guest users. They also perform an access review for inactive users, targeting 100 of the same guests as above, plus a different set of 200 guests. Billing: For March, Fabrikam is billed $375. This includes $225 for the lifecycle workflow (300 users x $0.75) and $150 for the access review (200 users x $0.75). Note that since 100 of the guest users already incurred a charge for the lifecycle workflow, they did not incur any additional charge for the inactive user review, since the maximum monthly charge per guest user is $0.75. April:  From the second group of 200 guest users in March, 150 guests receive an auto-assigned access package that grants access to an app and have the same inactive user access review that was run in March, repeated in April. Billing: For April, Fabrikam is billed $112.50 for the access review (150 users x $0.75), and they are not charged for the access package auto-assignment action, since these 150 guests have already been charged for April. Planning  To prepare for the upcoming changes, we recommend checking out Identity Governance features that could involve guests, including:  Auto-assigning policies in Entitlement Management Entitlement management policies using custom extensions, Verified ID, or Lifecycle Workflows Access reviews for user-to-group affiliation, inactive users, or PIM for groups  Also, check out all the audit logs for these activities to find all guest users who have had governance actions taken. We hope this update provides the clarity you need as we approach the launch of the ID Governance for guests add-on. While billing is not active yet, you can still use ID Governance for guests for free while it’s in public preview: https://aka.ms/IDGovernanceGuests     Learn more about Microsoft Entra   Prevent identity attacks, ensure least privilege access, unify access controls, and improve the experience for users with comprehensive identity and network access solutions across on-premises and clouds.  ⁠Microsoft Entra News and Insights | Microsoft Security Blog Microsoft Entra blog | Tech Community Microsoft Entra documentation | Microsoft Learn Microsoft Entra discussions | Microsoft Community  

Wow, what an incredible year of updates, new innovations, announcements, and of course, time with you all-our valuable customers, partners, and OneDrive users to gain valuable feedback that helps drive our product to be great! I wanted to wrap up the year with highlights of some of our key moments to and share resources for you to indulge in. Reflecting on OneDrive's Recent Innovations As we look back on our journey this year, we’ve introduced many features and updates to the OneDrive experience. From AI-powered capabilities to improved user interfaces, these updates help you work faster and more efficiently. Some of those highlights include:   Copilot in OneDrive (Web) One of the exciting features we announced was the general availability of Copilot in OneDrive for web. Copilot is there to help you work faster and smarter by generating summaries for large documents, comparing differences between multiple files, and answering complex questions using stored data. Whether generating ideas for new documents or quickly finding insights, Copilot became a valuable productivity companion.   Copilot in OneDrive: Comparing files in a table   Faster performance online or offline We've taken enhanced OneDrive's web experience for our users. Now, you can view, edit, and organize your files at lightning speed—even when you're offline. Whether you're browsing through large libraries or filtering documents, everything feels faster, smoother, and more responsive.   Colored Folders on Windows We introduced colored folders in Windows File Explorer, bringing a splash of color to file organization. You can customize folders with different colors, making it easier to organize and locate files. Whether managing personal projects or work documents, colored folders added a touch of personalization to file management.   Colored folders in Windows File Explorer   Sync Admin Report Export Capability (Public Preview) For our IT admins, we rolled out the Sync Admin report export capability in public preview this year. Enabling you to export detailed sync reports, gaining valuable insights into your sync performance and troubleshooting sync-related concerns more effectively.   Enhanced OneDrive Search experience We understand that searching for specific files among a mountain of documents can be time consuming. That's why we introduced a suite of enhancements to OneDrive, designed to optimize the search experience. These updates made it quicker and easier for you to locate the files you need, making the whole process faster and more efficient than ever before.   OneDrive’s next generation improvements As part of our continued commitment to bring the best OneDrive experience to you all, we introduced new enhancements to work better across M365 apps. The integration of the OneDrive app in Outlook allows you to access your OneDrive files directly from Outlook, making it easier to share and collaborate on documents without leaving your email. And we can’t forget about Media view helping you find and enjoy your media files easier than ever. Whether you're organizing your photos or reviewing videos, this streamlined experience keeps your media content at your fingertips.   OneDrive Media view   These updates marked significant milestones in our journey to enhance the OneDrive experience. We are looking forward to continuing to innovate and bring you more exciting features and improvements in the near future! OneDrive Event-AI Innovations for a New Era of Work and Home This past October we had our second annual OneDrive event where we unveiled a powerful lineup of new features designed to help you work smarter, stay organized, and relive life’s best moments. OneDrive Event Visual We introduced OneDrive agents custom AI assistants, created by you and grounded in your rich content that exists in your files. We also got a glimpse of OneDrive's improved search capabilities and faster performance, both online and offline. New features and AI-driven features to improve security and management. For our customers who use OneDrive in their personal lives, we shared a look at the new OneDrive mobile app- a vibrant, photos-first experience. With a refreshed UI, users can rediscover, search, and share their favorite photo memories easier than ever. When users back up their phone’s photos to OneDrive, they can enjoy AI-curated memories of meaningful moments. Sit back, grab a snack, and see all these announcements and more! Microsoft OneDrive: AI Innovations for a New Era of Work and Home OneDrive Office Hours One of our favorite monthly occurrences from our product teams is our chance to connect with you directly. Each month we have hosted webinars on multiple special topics across OneDrive and leave time for great conversations and Q&A at the end.   Customer Office Hours promo visual If you’ve missed any of this year’s session here is a list of our OnDemand content ready for your viewing pleasure: August-Copilot in OneDrive September-File Picker: seamless collaboration and file management October-Unlicensed OneDrive user accounts November-New Sync updates To stay updated on the latest Office Hours: Follow our blogs on Tech community Download the recurring calendar invite so you don’t miss out! Save the link https://aka.ms/OneDriveOfficeHours to register every month. Sync-Up Podcast Whether you love hearing about deep dive conversations with members of our product teams or seeing behind the scenes for some of our big events, the Sync Up Podcast has got you covered. Sync Up Podcast Episodes: OneDrive's Year in Review & Unlicensed User Changes In Focus—Designing for Copilot Data on the Move—Migrating to OneDrive From Waterfalls to Weekly Releases—Engineering Excellence with Steven Bailey and John Selbie Listen on your favorite platforms: Show: https://aka.ms/SyncUp | Apple Podcasts: https://aka.ms/SyncUp/Apple | Spotify: https://aka.ms/SyncUp/Spotify | RSS: https://aka.ms/SyncUp/RSS Your feedback is valuable to us! Please take our quick survey to suggest topics you'd like to see! A New Year of Innovation As we wrap up this incredible year, it was amazing to reflect on all the progress we've made together. From the introduction of AI-powered features to enhanced user experiences, OneDrive has truly evolved thanks to your feedback and support. Looking ahead, we're excited about the future and the innovations yet to come. We can't wait to continue this journey with you, bringing even more enhancements and features that will make your OneDrive experience better than ever. Thank you for being a part of our community and for your continued support. Here's to another year of growth, innovation, and success 🚀!

Please refer to my repo to get more AI resources, wellcome to star it: https://github.com/xinyuwei-david/david-share.git This article if from one of my repo: https://github.com/xinyuwei-david/david-share/tree/master/Deep-Learning/DPO-DeepSpeed-FSDP Direct Preference Optimization (DPO) is currently one of the popular methods for aligning large language models (LLMs) with human preferences. With parameter-efficient fine-tuning techniques like LoRA and QLoRA, we can perform DPO training on bigger models. Distributed training technology To train bigger model with 2 H100, we could use PyTorch's Fully Sharded Data Parallel (FSDP) technology, combined with parameter-efficient fine-tuning methods like LoRA and QLoRA. FSDP is similar to DeepSpeed's ZeRO technology. Accelerate is a library from Hugging Face (HF). FSDP is a distributed training technique that shards the model's parameters, optimizer states, and gradients, distributing them across multiple devices (such as GPUs). During the forward and backward passes, only the required parameter shards are loaded into memory and released after computation. This greatly reduces memory requirements. Of course, when training even larger models, DeepSpeed can be used. DeepSpeed requires a large amount of memory to store full-precision model parameters. In my repo, I used both DeepSpeed ZeRO-3 technology and FSDP technology, and the training results were the same. I will showcase the scripts and configuration files for both training methods. In the following DeepSpeed and Accelerate FSDP training, I use an adapter from HF DeepSpeed Training Deepspeed Configuration file, deepspeed_config.json: { "zero_optimization": { "stage": 3, "overlap_comm": true, "contiguous_gradients": true, "reduce_bucket_size": 104857600, "stage3_prefetch_bucket_size": 104857600, "stage3_param_persistence_threshold": 1048576 }, "bf16": { "enabled": true }, "train_micro_batch_size_per_gpu": 1, "gradient_accumulation_steps": 16, "steps_per_print": 10, "wall_clock_breakdown": false } Training code, deepspeed.py: import torch import os import multiprocessing from datasets import load_dataset from peft import PeftModel from transformers import ( AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, set_seed ) from trl import DPOTrainer, DPOConfig set_seed(1234) model_name = "Qwen/Qwen2.5-72B-Instruct" sft_adapter = "./adpter/" # 一个使用 SFT 微调的 LoRA 适配器 compute_dtype = torch.bfloat16 # 如果在使用 FlashAttention 时遇到问题，可以改用 'sdpa' attn_implementation = 'flash_attention_2' # 如果内存不足，可以修改以下三个训练参数 bs = 1 # 每个设备的批大小（训练和验证） gas = 16 # 梯度累积步骤数 mseqlen = 512 # 最大序列长度 lr = 1e-5 # 学习率 QLoRA = True # 是否量化基模型 output_dir = "./DPO" # 初始化 Tokenizer tokenizer = AutoTokenizer.from_pretrained(model_name) tokenizer.pad_token = "<|image_pad|>" tokenizer.pad_token_id = 151655 tokenizer.padding_side = 'right' # 对于 Qwen2.5，左右 padding 都可以 # 加载并处理数据集 ds = load_dataset("mlabonne/orpo-dpo-mix-40k", split="train").train_test_split(test_size=0.01) ds_train = ds['train'] ds_test = ds['test'] def process(row): # 第一个消息是提示 prompt_messages = tokenizer.apply_chat_template([row["chosen"][0]], tokenize=False) chosen_messages = tokenizer.apply_chat_template(row["chosen"][1:], tokenize=False) + tokenizer.eos_token rejected_messages = tokenizer.apply_chat_template(row["rejected"][1:], tokenize=False) + tokenizer.eos_token row["prompt"] = prompt_messages row["chosen"] = chosen_messages row["rejected"] = rejected_messages return row ds_train = ds_train.map( process, num_proc=multiprocessing.cpu_count(), load_from_cache_file=False, ) ds_test = ds_test.map( process, num_proc=multiprocessing.cpu_count(), load_from_cache_file=False, ) if QLoRA: bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=compute_dtype, bnb_4bit_use_double_quant=True, bnb_4bit_quant_storage=compute_dtype, ) model = AutoModelForCausalLM.from_pretrained( model_name, quantization_config=bnb_config, torch_dtype=compute_dtype, attn_implementation=attn_implementation, ) # 冻结基模型的参数 for name, param in model.named_parameters(): param.requires_grad = False # 让输入嵌入支持梯度 def make_inputs_require_grad(module, input, output): output.requires_grad_(True) model.get_input_embeddings().register_forward_hook(make_inputs_require_grad) else: model = AutoModelForCausalLM.from_pretrained( model_name, torch_dtype=compute_dtype, attn_implementation=attn_implementation, ) model.gradient_checkpointing_enable(gradient_checkpointing_kwargs={'use_reentrant': True}) # 加载 LoRA 适配器 model = PeftModel.from_pretrained( model, sft_adapter, is_trainable=True, adapter_name="DPO" ) model.load_adapter(sft_adapter, adapter_name="reference") # 将模型移动到设备上 device = torch.device("cuda" if torch.cuda.is_available() else "cpu") model.to(device) training_arguments = DPOConfig( output_dir=output_dir, eval_strategy="steps", do_eval=True, optim="adamw_torch", per_device_train_batch_size=bs, gradient_accumulation_steps=gas, per_device_eval_batch_size=bs, log_level="debug", save_strategy="steps", save_steps=5, logging_steps=2, learning_rate=lr, bf16=True, beta=0.1, eval_steps=2, max_steps=10, warmup_ratio=0.1, lr_scheduler_type="linear", max_length=mseqlen, max_prompt_length=512, dataset_num_proc=multiprocessing.cpu_count(), model_adapter_name="DPO", ref_adapter_name="reference", deepspeed="deepspeed_config.json", # 指定 DeepSpeed 配置文件 ) trainer = DPOTrainer( model=model, args=training_arguments, train_dataset=ds_train, eval_dataset=ds_test, tokenizer=tokenizer, ) # 开始训练 trainer.train() # 保存模型 trainer.save_model(output_dir) Launch training: (dpo) root@h1002gpu:~# deepspeed deepspeed.py Training result analyze In DPO training, the model is provided with a set of conversations, each containing the same "prompt" or "question", along with corresponding "chosen" and "rejected" replies. The model needs to learn to distinguish between these replies and prefer generating high-quality "chosen" responses. Training data and results The training data includes: Source: Airoboros Chosen Reply: Contains multiple rounds of dialogue Rejected Reply: Contains multiple rounds of dialogue Prompt: A descriptive text Question: The same text as the prompt Sometimes in the data, the "prompt" and "question" may be identical, which can serve as the starting point for the conversation in certain training settings. Training results are as following: Next, I will combine the training data to roughly introduce the DPO training process and results. DPO training process and results explanation Core Objective of DPO Objective: Directly optimize the model parameters to reflect human preferences without the need for a separate reward model. DPO uses human preference data to adjust the model directly, making its generated responses more aligned with human expectations. Introducing the Reference Model: To prevent the model from deviating from its original language capabilities during optimization, DPO introduces a reference model (usually a copy of the initial model with fixed parameters) as a regularization term. Maintaining Language Capabilities: The reference model provides a baseline of the model before adjustment. By comparing with the reference model, the trained model can learn human preferences while avoiding overfitting and deviation from its original abilities, ensuring that its language understanding and generation capabilities remain intact. This helps prevent the model from prioritizing human preferences at the expense of core language skills like grammatical correctness and factual accuracy. Role of the Reference Model: Training Data Prompt: User input, for example: "Please explain the phase changes of water." Chosen Reply: Responses evaluated by humans as high-quality, fully answering the question, and meeting expectations. These replies are typically accurate, complete, relevant, and fluent, satisfying user needs. Rejected Reply: Responses evaluated by humans as lower quality, not adequately answering the question, or not meeting expectations. These replies may lack accuracy, contain incomplete information, be irrelevant to the prompt, or be less fluent. Human Evaluation Criteria: Accuracy: Is the content of the reply correct and free from misleading information? Completeness: Does the reply fully answer the user's question? Relevance: Is the reply closely related to the user's prompt? Fluency: Is the reply grammatically correct and clearly expressed? Example: Prompt: "Please explain the phase changes of water." Chosen Reply: Evaluation Reasoning: The reply accurately explains the process of water's phase changes, provides complete information, is highly relevant to the prompt, and is fluent. Water exists in three states: solid, liquid, and gas. Through changes in temperature and pressure, water can transition between these states. For example, ice (solid) melts into water (liquid) when heated, and water vaporizes into steam (gas) upon further heating. Rejected Reply: Evaluation Reasoning: The reply does not address the question about the phase changes of water; the information is incomplete, and the relevance is insufficient. Water is a very common substance found everywhere in daily life. Training Process Step 1: Calculate Log Probabilities For the trained model (parameters θ): Log probability of the chosen reply: log_p_model(chosen | prompt) = log( π_θ(chosen | prompt) ) Log probability of the rejected reply: log_p_model(rejected | prompt) = log( π_θ(rejected | prompt) ) For the reference model (fixed parameters): Log probability of the chosen reply: log_p_ref(chosen | prompt) = log( π_ref(chosen | prompt) ) Log probability of the rejected reply: log_p_ref(rejected | prompt) = log( π_ref(rejected | prompt) ) Step 2: Calculate Preference Differences Preference difference for the chosen reply: Δ_chosen = log_p_model(chosen | prompt) - log_p_ref(chosen | prompt) Preference difference for the rejected reply: Δ_rejected = log_p_model(rejected | prompt) - log_p_ref(rejected | prompt) Step 3: Construct the Loss Function Loss function:Where β is the temperature hyperparameter controlling sensitivity to preference differences. loss = -log( exp(Δ_chosen / β) / [ exp(Δ_chosen / β) + exp(Δ_rejected / β) ] ) Objective: Minimize the loss function loss to make the model more inclined to generate the "chosen" reply over the "rejected" reply. Training Process Example Assumed Values (for Illustration): log_p_model(chosen | prompt) = -5 log_p_model(rejected | prompt) = -7 log_p_ref(chosen | prompt) = -6 log_p_ref(rejected | prompt) = -6 Calculate Preference Differences: Δ_chosen = (-5) - (-6) = 1 Δ_rejected = (-7) - (-6) = -1 Calculate the Loss Function (assuming β = 1): Calculate the numerator: exp(Δ_chosen / β) = exp(1) ≈ 2.718 Calculate the denominator: exp(Δ_chosen / β) + exp(Δ_rejected / β) = exp(1) + exp(-1) ≈ 2.718 + 0.368 ≈ 3.086 Calculate the loss: loss = -log( 2.718 / 3.086 ) = -log(0.880) ≈ 0.127 Result Analysis: The loss value is relatively small (approximately 0.127), indicating that the model tends to prefer the "chosen" reply. Optimize Model Parameters: Through backpropagation, minimize the loss function loss to further enhance the model's preference for the "chosen" reply. Explanation of Training Log Fields Based on the DPO training process, here's a detailed explanation of each field in the training log and their importance in evaluating training effectiveness: Example Training Log: { 'loss': 0.6931, 'grad_norm': 0.05, 'learning_rate': 1e-5, 'rewards/chosen': 0.0, 'rewards/rejected': 0.0, 'rewards/accuracies': 0.5, 'rewards/margins': 0.0, 'logps/chosen': -15.0, 'logps/rejected': -15.0, 'logits/chosen': [0.2, 0.3, ...], 'logits/rejected': [0.2, 0.3, ...], 'epoch': 0 } 1. loss Meaning: Represents the loss value at the current training step, measuring the model's ability to distinguish between the "chosen" and "rejected" replies. Importance: Core Indicator: The primary metric to evaluate training effectiveness. Training Goal: Minimizing loss indicates successful learning toward preferring the "chosen" reply. Indicator Trend: Initial Stage: loss is typically higher (around 0.6931), indicating no preference. During Training: Should decrease over time, showing the model is learning to prefer the "chosen" reply. 2. grad_norm Meaning: Represents the gradient norm of the model parameters, indicating the overall magnitude of parameter updates. Importance: Learning Intensity: Reflects how much the model is adjusting its parameters. Training Stability: Helps detect issues like vanishing or exploding gradients. Indicator Trend: Normal Range: Should be within a reasonable range (e.g., 0.01 to 1). Abnormal Situations: Too Small: Near zero may indicate lack of learning. Too Large: May require gradient clipping to prevent instability. 3. learning_rate Meaning: Controls the step size in parameter updates during training. Importance: Convergence Speed and Stability: Affects how quickly and smoothly the model learns. Adjustment Strategy: Slow Loss Decrease: Consider increasing the learning rate. Unstable Training: If loss fluctuates, decreasing the learning rate might help. 4. rewards/chosen and rewards/rejected Meaning: rewards/chosen: Reward value for the "chosen" reply (Δ_chosen). rewards/rejected: Reward value for the "rejected" reply (Δ_rejected). Importance: Model Preference: Indicates the model's inclination towards each reply. Indicator Trend: Initial Stage: Both may be around 0.0 (no preference). During Training: rewards/chosen should increase. rewards/rejected should decrease. 5. rewards/accuracies Meaning: The proportion of times the model correctly prefers the "chosen" reply. Importance: Performance Measure: Directly evaluates preference learning. Indicator Trend: Initial Stage: Around 0.5 (random guess). During Training: Should approach 1.0, indicating improved preference accuracy. 6. rewards/margins Meaning: The difference between rewards/chosen and rewards/rejected. rewards/margins = rewards/chosen - rewards/rejected Importance: Discrimination Ability: Larger margins indicate better distinction between replies. Indicator Trend: Should increase during training. 7. logps/chosen and logps/rejected Meaning: Total log probabilities of generating the "chosen" and "rejected" replies. Importance: Probability Basis: Used in calculating preference differences and rewards. Indicator Trend: Increasing logps/chosen indicates higher probability for the "chosen" reply. Stable or decreasing logps/rejected shows reduced preference for the "rejected" reply. 8. logits/chosen and logits/rejected Meaning: Raw output scores from the final layer before applying softmax, for both replies. Importance: Probability Calculation: Used to compute probabilities for each token, affecting log probabilities. Indicator Trend: Ensure Valid Values: Avoid nan or inf values. Monitor Changes: Changes in logits reflect learning progress. 9. epoch Meaning: Indicates the current training epoch or iteration over the training dataset. Importance: Training Progress: Helps track how far along the training is. Indicator Trend: As epoch increases, expect improvements in other metrics. Summary Adjust Training Strategies Based on Indicators: Slow Loss Decrease: Increase learning rate or check data quality. Gradient Issues: If grad_norm is abnormal, inspect gradient computations or adjust optimizer settings. Low Preference Accuracy: Enhance data quality or quantity. Small Reward Margins: Adjust the temperature parameter β to influence sensitivity. Emphasize the Importance of the Reference Model: Maintaining Language Capabilities: Ensures the model doesn't overfit human preferences at the cost of language understanding and generation skills. Balancing Objectives: Optimizes for human preference while retaining overall model performance. Continuous Monitoring and Adjustment: Regular Evaluation: Use a validation set to assess performance and prevent overfitting. Dynamic Adjustment: Modify training strategies based on log indicators to optimize the model. By understanding DPO's core concepts, training processes, and how to interpret key training metrics, you can effectively train a model that aligns with human preferences while maintaining strong language capabilities.

Many applications offer chat with automated capabilities but lack the depth to fully understand and address user needs. What if a chat app could not only connect people but also improve conversations with AI insights? Imagine detecting customer sentiment, bringing in experts as needed, and supporting global customers with real-time language translation. These aren’t hypothetical AI features, but ways you can enhance your chat apps using Azure Communication Services and Azure OpenAI today. In this blog post, we guide you through a quickstart available on GitHub for you to clone and try on your own. We highlight key features and functions, making it easy to follow along. Learn how to upgrade basic chat functionality using AI to analyze user sentiment, summarize conversations, and translate messages in real-time. Natural Language Processing for Chat Messages First, let’s go through the key features of this project. Chat Management: The Azure Communication Services Chat SDK enables you to manage chat threads and messages, including adding and removing participants in addition to sending messages. AI Integration: Use Azure OpenAI GPT models to perform: Sentiment Analysis: Determine if user chat messages are positive, negative, or neutral. Summarization: Get a summary of chat threads to understand the key points of a conversation. Translation: Translate into different languages. RESTful endpoints: Easily integrate these AI capabilities and chat management through RESTful endpoints. Event Handling (optional): Use Azure Event Grid to handle chat message events and trigger the AI processing. The starter code for the quickstart is designed to get you up and running quickly. After entering your Azure Communication Services and OpenAI credentials in the config file and running a few commands in your terminal, you can observe the features listed above in action. There are two main components to this example. The first is the ChatClient, which manages the capturing and sending of messages, via a basic chat application using Azure Communication Services. The second component, OpenAIClient, enhances your chat application by transmitting messages to Azure OpenAI along with instructions for the desired types of AI analysis. AI Analysis with OpenAIClient Azure OpenAI can perform a multitude of AI analyses, but this quickstart focuses on summarizing, sentiment analysis, and translation. To achieve this, we created three distinct prompts for each of the AI analysis we want to perform on our chat messages. These system prompts serve as the instructions for how Azure OpenAI should process the user messages. To summarize a message, we hard-coded a system prompt that says, “Act like you are an agent specialized in generating summary of a chat conversation, you will be provided with a JSON list of messages of a conversation, generate a summary for the conversation based on the content message.” Like the best LLM prompts, it’s clear, specific, and provides context for the inputs it will get. The system prompts for translating and sentiment analysis follow a similar pattern. The quickstart provides the basic architecture that enables you to take the chat content and pass it to Azure OpenAI for analysis.   , and summarization. The Core Function: getChatCompletions The getChatCompletions function is a pivotal part of the AI chat sample project. It processes user messages from a chat application, sends them to the OpenAI service for analysis, and returns the AI-generated responses. Here’s a detailed breakdown of how it works: Parameters The getChatCompletions function takes in two required parameters: systemPrompt: A string that provides instructions or context to the AI model. This helps guide OpenAI to generate appropriate and relevant responses. userPrompt: A string that contains the actual message from the user. This is what the AI model analyzes and responds to. Deployment Name: The getChatCompletions function starts by retrieving the deployment name for the OpenAI model from the environment variables. Message Preparation: The function formats and prepares messages to send to OpenAI. This includes the system prompt with instructions for the AI model and user prompts that contain the actual chat messages. Sending to OpenAI: The function sends these prepared messages to the OpenAI service using the openAiClient’s getChatCompletions method. This method interacts with the OpenAI model to generate a response based on the provided prompts. Processing the Response: The function receives the response from OpenAI, extracts the AI-generated content, logs it, and returns it for further use. Explore and Customize the Quickstart The goal of the quickstart is to demonstrate how to connect a chat application and Azure OpenAI, then expand on the capabilities. To run this project locally, make sure you meet the prerequisites and follow the instructions in the GitHub repository. The system prompts and user messages are provided as samples for you experiment with. The sample chat interaction is quite pleasant. Feel free to play around with the system prompts and change the sample messages between fictional Bob and Alice in client.ts to something more hostile and see how the analysis changes. Below is an example of changing the sample messages and running the project again. Real-time messages For your chat application, you should analyze messages in real-time. This demo is designed to simulate that workflow for ease of setup, with messages sent through your local demo server. However, the GitHub repository for this quickstart project provides instructions for implementing this in your actual application. To analyze real-time messages, you can use Azure Event Grid to capture any messages sent to your Azure Communication Resource along with the necessary chat data. From there, you trigger the function that calls Azure OpenAI with the appropriate context and system prompts for the desired analysis. More information about setting up this workflow is available with "optional" tags in the quickstart's README on GitHub. Conclusion Integrating Azure Communication Services with Azure OpenAI enables you to enhance your chat applications with AI analysis and insights. This guide helps you set up a demo that shows sentiment analysis, translation, and summarization, improving user interactions and engagement. To dive deeper into the code, check out the Natural Language Processing of Chat Messages repository, and build your own AI-powered chat application today!  

Recently I had a scenario where I needed to test AS2 Msg Send/receive with encryption & Decryption. I realized there a lots of good guidance samples are available but they were in lots of different places and had some dependencies. I am documenting the steps I went through to make the sample work.   Step 1: To start this is great sample to try the whole end to end scenario and very easy to deploy. Only thing missing is Encryption and decryption. https://github.com/Azure/azure-quickstart-templates/tree/master/quickstarts/microsoft.logic/logic-app-as2-send-receive Once you have deployed the solution which has all the Integration Accounts, Partners, Agreements and Logic Apps to Send and receive the messages, test out the solution. Now you need your certificates to Encrypt and decrypt your messages. You can use Public Certificates purchased from Certificate Authorities and they don't require any Keys If you plan to use self-signed certificates which was my case, you will need Private Key in Azure Key Vault.   Step 2: Create Azure Key Vault Note: Make sure you give appropriate access to "Azure Logic Apps" to Key Vault using Access Policies or now preferred RBAC approach.   Step 3: Create Self Signed Certificate (for testing purposes only) If you already have the public Certificate and Private Key, you can upload the private Key in the Key Vault. Step 4: Go to the Contoso Integration Account (receiving party) and add the Private Key to the integration account You might encounter an error or wont see any Keys when you select the KeyVault. This happens if you have not given access to the Azure Logic Apps to Key Vault.   Click on Add to add the Private Key   Step 5: Go to Agreements and click on edit button to edit the Contoso-FabrikamSales agreement. Click on Receive Settings and select Enable Message Encryption. Select the Private Key added in earlier step in the drop down. Step 6: Make sure Contoso AS2Receive Logic Apps has the decrypt Step/Action   Step 7: Go to the Fabrikam Integration Account (sending party) and add the public certificate to the integration account. Step 8: Go to Agreements and click on edit button to edit the FabrikamSales-Contoso agreement. Click on Send Settings and select Enable Message Encryption. Select the Public Certificate added in earlier step in the drop down.   Step 9: Go to your Send Logic App and Configure the Encode Step Action. Make sure you populate the AS2 From and AS2 To Parameter with correct Values.   Step 10: Go to Http Action add below parameters For Header Body('Encode_to_AS2_message')?['AS2Message']?['OutboundHeaders']" For Body @base64ToBinary(body('Encode_to_AS2_message')?['AS2Message']?['Content']) Make Sure You have correct URI to the Receiving Logic App. Step 11: Last step is to test the flow. Go to FabrikamSales-AS2Send logic app and run it manually. Validate if it successfully executed. Go to Contoso-AS2Receive and confirm if it also ran successfully. GO to the Run History identifier click on it and check the Decode AS2 Msg action Output and check dispositionType is not giving any errors.   References: Add certificates to secure B2B messages in workflows - Azure Logic Apps | Microsoft Learn Exchange AS2 messages in B2B workflows - Azure Logic Apps | Microsoft Learn Azure Logic Apps - AS2 Send Receive - Code Samples | Microsoft Learn Solutions for common errors and problems in B2B scenarios - Azure Logic Apps | Microsoft Learn  

Introduction Integrating Azure Machine Learning with Azure SQL Managed Instance supports use cases such as feature engineering, model training, and the development of machine learning models using data from Azure SQL MI. This integration enables comprehensive insights and data-driven decision-making. However, deploying Azure SQL MI in reserved IP spaces like 172.17.x.x/16, which is a commonly used IP address range, can cause conflicts in customer environments, disrupting integration. This blog post provides solutions to overcome these challenges and seamlessly integrate Azure ML with Azure SQL MI. Understanding the Constraint Azure ML internally utilizes 172.17.0.0/16 IP address range for the Docker bridge network. This commonly used IP address range can cause errors and integration issues. Refer to the official documentation for additional information. Strategies for Resolving the Integration Challenge 1. IP Address Range Modification One straightforward solution, if possible, is to modify the IP address range of your Azure SQL MI deployment. By choosing a different address space that does not conflict with the commonly used 172.17.x.x/16 IP address range, can help avoid potential conflicts. 2. Using Private Endpoints An alternative approach is using Private Endpoints with Azure SQL Managed Instance (MI), configured with an address space other than 172.17.x.x range. This blog post explores how private endpoints enable seamless integration. Steps to resolve integration challenge using Private Endpoint To address the challenge of integrating Azure ML with Azure SQL MI, which is already deployed in the 172.17.x.x/16 address range, follow these steps: Add additional address space to the VNET used in Azure SQL MI Establish a Private Endpoint Connection Configure the Private DNS Zone The reference target architecture for the solution is as below.   The key characteristics of the reference target architecture are, Adopts the recommended Hub-Spoke architecture, enhancing scalability, security, centralized management, and overall network performance. The Network Virtual Appliance (NVA) used in this example is Azure Firewall and is deployed in the Hub. Route table is used to configure the traffic flow between Azure ML & Azure SQL MI. Let us examine the steps required to address the integration challenge in further detail. Add additional address space to the VNET used in Azure SQL MI In the VNET used by Azure SQL MI, add an additional address space apart from 172.17.x.x/16. For example, you can add 173.17.0.0/24 address space to the VNET. The next step is to add a subnet to this address space. This subnet can be used when private endpoint is created. Note, there is no change to the ManagedInstance subnet which is used by Azure SQL MI. In this example, it is deployed in 172.17.2.0/27. Establish a Private Endpoint Connection We will utilize the private endpoint connection created in Azure SQL MI and use the same to connect from Azure ML. Thanks to Jose Manuel Jurado’s blog, which provides the steps for creating Private Endpoint connection for Azure SQL MI. Create the private endpoint using the newly created address space & subnet. Resource for the private endpoint should be managedInstance. The virtual network and subnet should be the vnet where Azure SQL MI is deployed and subnet should be the subnet created in the previous step. The DNS zone will be configured later. Configure the Private DNS Zone The next step is to create private DNS zone for Azure SQL MI. The naming convention for the private DNS zone is privatelink.<<DNS Zone>>.database.windows.net. The DNS Zone in the above naming convention is derived from the hostname of Azure SQL MI. For example, if the host name is sqlmiinstance.2292572b.database.windows.net then the DNS Zone is 2292572b. In this example, the Private DNS Zone to be created will be privatelink.2292572b.database.windows.net. After creating the private DNS zone, update the DNS zone configuration in the previously created private endpoint and associate it with the new private DNS zone. In the Private DNS zone, add the spoke VNETs in this case, the vnets used by Azure SQL MI and Azure ML in the Virtual Network Links. This will enable DNS of Azure SQL MI from these Virtual Networks. Testing To test the configuration, create a compute instance in Azure ML and attempt to connect to the Azure SQL MI from the same. The first test is to verify if the compute instance is able to reach Azure SQL MI. As shown below, Private DNS successfully resolves the FQDN of Azure SQL MI to the correct IP address. The resolved IP address is 173.17.x.x, which corresponds to the additional address space and subnet created in the earlier step. This ensures there is no conflict with the 172.17.x.x range internally used by Azure ML. Second test is to connect to Azure SQL MI and run a query. With this configuration, the connection to Azure SQL ML is successfully established and the specific database can be accessed. Conclusion Integrating Azure ML with Azure SQL MI deployed in the 172.17.x.x/16 address space can pose challenges, but with thoughtful planning and the right strategies, these constraints can be effectively resolved. Using Private Endpoints can be an effective solution to resolve the issue when modifying IP address range is not feasible. By addressing the integration challenge, you can leverage the full potential of Azure ML & Azure SQL MI and create a robust & scalable environment for your applications.

At this year’s Ignite, we announced new benefits in ISV Success designed to support software companies as they build new AI powered apps and publish to the commercial marketplace. During this session, we also spoke with Mike Mason, Global Alliance Director from Varonis, as he shared how they have leaned into the commercial marketplace to grow their business.   For additional details on eligibility for the new and expanded benefits, check out our previous post.     Technical guidance to help you get started   The team highlighted new opportunities for technical guidance focused on building AI powered solutions, including new AI Envisioning events. AI Envisioning Days offer sessions for both business and technical teams that help participants identify generative AI use cases and then build with a proven development framework. These events kicked off in November and are held monthly across 3 time zones. To take advantage of these sessions, you can register here.    In addition to these events, ISV Success now offers software companies 1:1 AI consults tailored to their specific app development needs.     Developer tools & resources to get to market faster   ISV Success participants can now use Azure credits towards GitHub Copilot to accelerate development. GitHub Copilot is the most widely adopted AI development tool built by GitHub and OpenAI and offers code completion and automatic programming to help with repetitive tasks.   Additionally, ISVs with certified software designations are now able to access financial incentives through ISV Success and Marketplace Rewards. These incentives are available through the new Advanced Package and offer up to $150K for AI and analytics projects and up to $50K to migrate end-customers to an ISV’s Azure-based solution.     Richer benefits and accelerated rewards for AI projects  ISVs currently building AI apps and intending to publish a transactable offer on the marketplace now qualify for the ISV Success expanded package. This package provides $25K in Azure to offset development costs and 50 hours of 1:1 technical consults. Once published, these ISVs will have the ability to unlock GTM benefits through Marketplace Rewards earlier, which include customer propensity scoring and Azure sponsorship.   Best practices from Varonis’ Mike Mason  During an interview on stage, Mike shared his advice for other software companies looking to grow their business on the Microsoft commercial marketplace.   Meet customers where they are   Varonis first moved to transact on marketplace over two years ago for a customer that wanted to use their Microsoft Azure Consumption Commitment (MACC) to purchase Varonis’ solution. Customers continue to indicate that they prefer the seamless procurement process and single invoice offered through marketplace transactions.   Bring channel partners along   Varonis is a channel first business and has participated in the marketplace MPO launches across the US and UK. For ISVs looking to sell alongside their partners in marketplace, Microsoft maintains an MPO list of eligible partners. If an ISV finds that their partner is not yet listed, they can email channelready@microsoft.com to get that partner added.   Internal alignment and enablement is key   Varonis has strategically brought along teams across the business – including Finance, Operations, Marketing and Sales – for a unified approach to marketplace that has contributed to their success.    As the Global Alliance Director, Mike travels to each region and shares marketplace results and success stories.   To watch the full “What’s new in ISV Success – AI benefits for software companies and more” session from Ignite 2024 click here.    To learn more about the benefits of ISV Success, check out the ISV Hub.   Ready to enroll in ISV Success? Visit the sign-up form.        

We have released an update for both Mixed Reality Link (24.12.17013.0) and Windows App on Quest (10.0.19.1305).    The changes in this update are:   -Improved stability -Expanded language support to include Danish, Finnish, Norwegian, Polish, Portuguese and Swedish languages. -Improved performance

Microsoft commercial marketplace supports partners in addressing customers’ cloud needs through various deal-making scenarios. In the Ignite session titled “Activating the Channel Opportunity Through the Marketplace,” Alison Buggia, Principal Product Manager at Microsoft; Jason Rook, Senior Director of Product Marketing for the Commercial Marketplace at Microsoft; and Rob Phillips, Senior Director of Microsoft Sales at Presidio, explore how to activate the ecosystem with partner-to-partner selling opportunities. They share valuable insights into leveraging the Microsoft commercial marketplace to drive business growth.  Key Points:  Microsoft commercial marketplace is experiencing significant growth, with over 100% year-over-year growth in marketplace-billed sales, a 43% growth in the number of ISV solutions that are IP co-sell eligible, and 110% growth in customer cloud commitment spend. This indicates a robust and expanding platform that offers substantial opportunities for partners.  There are three primary ways to unlock this growth and sell through private offers: CSP private offers, multiparty private offers, and professional services private offers. Each of these offers unique benefits and can be tailored to different business models and customer needs.  Benefits:  CSP Private Offers: CSP Private offers allow channel selling at scale. They help partners unlock scale, maintain customer billing relationships, and provide comprehensive solutions. This allows ISVs to work with partners in the Microsoft Cloud Solution Provider program and resell their products through them to reach more customers. It also allows Cloud Solution Provider partners to leverage partnerships with ISVs and continue to serve customers through the Microsoft commercial marketplace. This is particularly beneficial for reaching a global customer base and leveraging existing partnerships.  Multiparty Private Offers: Available in the United States, United Kingdom, and Canada, multiparty private offers help partners leverage cloud commitments and maintain customer relationships, leading to larger deals. This is a significant opportunity for partners to access pre-committed cloud budgets and drive substantial sales.  Professional Services Private Offers: Available in the United States, United Kingdom, and Canada, selling professional services through the Microsoft commercial marketplace creates new selling opportunities, simplifies invoicing and payment for customers, and enables partners to offer complete solutions on a single platform. This streamlines the sales process and enhances customer satisfaction by providing a more seamless procurement experience.  Marketplace Rewards Program: This program offers benefits like Azure Sponsorship, press release support, and customer case studies to help partners grow their marketplace business. These incentives can significantly boost your marketing efforts and enhance your visibility in the marketplace.  Opportunities for Growth:  The rapid growth of the marketplace presents a significant opportunity for partners to expand their customer base and increase sales. By leveraging private offers, you can reach more customers and simplify the sales process.  Multiparty private offers provide a substantial growth opportunity by allowing partners to harness the entire power of the Microsoft ecosystem and drive larger deals. This can lead to increased revenue and stronger customer relationships.  The Marketplace Rewards program provides additional incentives and support to help partners grow their business and achieve success. By participating in this program, you can access valuable resources and support to enhance your marketplace presence and drive sales.  Actionable Steps for ISVs:  Enroll in the Microsoft AI Cloud Partner Program: This is the first step to becoming eligible for private offers and accessing the benefits of the marketplace.  https://aka.ms/MAICPP  Get Transactable: Publish an app that is transactable in the marketplace and complete the necessary steps in Partner Center. This will enable you to start selling through the marketplace and leverage the benefits of private offers.  https://aka.ms/ISVSuccess  Focus on Azure IP Co-Sell Incentivized Status: Achieving this status will unlock co-sell opportunities and allow you to leverage cloud commitments.  Co-sell requirements - Partner Center | Microsoft Learn  Educate and Equip Your Sales Teams and Your Channel Partners: Ensure your sales teams understand the benefits of the marketplace and how to promote it to customers. This will help drive sales and increase customer engagement.  https://aka.ms/PartnerMPOCampaign  Leverage Marketplace Rewards: Take advantage of the benefits offered by the Marketplace Rewards program to enhance your marketing efforts and grow your business.  Manage marketplace rewards - Partner Center | Microsoft Learn    Actionable Steps for Channel Partners:  Enroll in the Microsoft AI Cloud Partner Program: This is the first step to becoming eligible for private offers and accessing the benefits of the marketplace.  https://aka.ms/MAICPP  Enroll in Commercial Marketplace and Multiparty Private Offers: Channel partners must enroll in Microsoft commercial marketplace then complete a tax and payment profile in Microsoft Partner Center.   Need help?  contact channelready@microsoft.com  https://aka.ms/MPO-channel-partner-onboarding  Leverage Marketplace Rewards: Take advantage of the benefits offered by the Marketplace Rewards program to enhance your marketing efforts and grow your business.  https://aka.ms/MarketplaceRewardsforChannelPartners    Leveraging the growth opportunities presented in the Microsoft commercial marketplace, you can accelerate growth for your business and achieve success in the digital marketplace.  You can watch the full Ignite session, “Activating the Channel Opportunity Through the Marketplace,” and learn more about driving growth through the marketplace. 

As customers move workloads to the cloud, designing their applications for high resiliency becomes a key consideration. This blog will highlight how Azure Load Balancer enables highly resilient applications across multiple resiliency categories. How to approach resiliency with Azure Load Balancer? When it comes to designing highly resilient applications, there are multiple types of design patterns that should be considered. Zone-redundancy Customers want to ensure that their deployments in an Azure region are resilient to any data center failures. Azure provides within-region resiliency via Azure Availability Zones. Availability Zones are separated groups of datacenters within a region. Customers can deploy their applications and Azure Load Balancer in a zone-redundant deployment method. A zone-redundant load balancer is replicated across all available zones. If one of the zones fails, then all incoming traffic will be sent to the other two zones, ensuring high availability.   Global resiliency For mission critical applications or applications that need a global presence, replicating them across multiple Azure regions will ensure the application is globally resilient. This design pattern will ensure that no single region or geography becomes a single point of failure. In case a region goes offline, traffic will failover to the next available region, ensuring high availability of the application. Azure global Load Balancer can then be used to distribute traffic to these multi-region applications. Azure global Load Balancer is a globally distributed layer-4 load balancing solution. Global load balancer provides features such as a globally static IP address, geo-proximity routing, and automatic failover to the next available region.     Subscription resiliency The last two categories of resiliency focused on ensuring the infrastructure itself is resilient. A third bucket focuses on resource management and isolation. Deploying resources and applications into multiple subscriptions can ensure that they are resilient to any subscription-level events. For example, if someone accidentally deletes a subscription or changes permissions, the overall application will still be available due to the replicated instances in the other subscriptions. To support multi-subscription workloads, Azure now supports cross-subscription load balancing. Now, the load balancer’s frontend IP address or backend pool can be located in a subscription that is different than the load balancer.   An example of a real world customer To better understand how all the categories of resiliency can be combined, let’s explore an example customer scenario. In this scenario, we will walk through the customer’s use-case and how Azure Load Balancer helped them deploy a highly resilient application. Who is the customer? In this scenario we will be learning about an example customer called Contoso. Contoso is a large retail company based in Europe, and they have a global presence. Contoso is moving off their on-prem environment and onto Azure to support their high-scale needs. What are the application requirements? As the team at Contoso is looking at moving their application to the cloud, they have strict requirements around resiliency that need to be addressed. As mentioned above, Contoso has a global presence and most of their applications need to be globally available. With that, Contoso will deploy replicas of an application across multiple Azure regions to ensure high-availability and resiliency for their global user base. Second, given the criticality of some applications (e.g., inventory manager), these applications need to be deployed in a redundant manner. As mentioned earlier, these applications will be replicated across multiple Azure regions, but Contoso needs these applications to also be replicated within a single Azure region. Third, each application deployment should be isolated from each other, and shouldn’t share a single resource (virtual machine, IP address, etc.). This requirement also extends to subscriptions, where each application replica will be isolated in its own subscription. In addition to the requirements outlined above, Contoso decided that Azure Load Balancer would be a perfect fit for their ingress needs, given its ultra-low latency capabilities. However, Contoso wanted to ensure that Azure Load Balancer would be able to meet their strict resiliency requirements. How did Azure Load Balancer help? To address Contoso’s resiliency requirements, they deployed Azure Load Balancer in a multi-tier architecture. To support their multi-region requirement, an Azure global load balancer was deployed. The global load balancer would then be the gateway into the overall application. Whenever traffic would be sent to the global endpoint, Azure would route it to the closest deployment to that user. In addition, global load balancer’s automatic health probes and failover capabilities gave Contoso peace of mind, knowing that in the off chance of a regional failure, traffic would be automatically routed to the other geos. To support their in-region redundancy requirements, Contoso adopted a zone-redundant architecture for all of their in-region infrastructure (virtual machines, IP addresses, load balancers, etc.). Finally, Contoso adopted the new cross-subscription load balancing feature. With this feature, Contoso can deploy each application replica in its own subscription and then link them to their Azure global Load Balancer. This allows each deployment to be independent and avoid a common failure point. Contoso, after adopting Azure Load Balancer, has developed an architecture that addresses resiliency across the stack. Furthermore, Contoso’s application is now resilient globally, regionally, and at the subscription layer.   Learn more Azure global Load Balancer Azure cross-subscription Load Balancer Zone-redundancy with Azure Load Balancer

Introducing Real-Time Multi-Modal Customer Service Solution Accelerator This blog post introduces a solution accelerator specifically designed to automate customer service using AI. This innovative tool brings several key capabilities to the table: Multi-Modal Real-Time Communication: The accelerator supports text chat and voice and has plans to incorporate video interactions, providing customers with a comprehensive communication suite. Scalable Framework: It offers a framework that can expand across various domains, simulating the expertise of multiple human agents in a real-world customer service environment. Microservice Architecture: With a stateful and scalable design, the architecture clearly separates the stateful agent service from the front-end layer, ensuring efficient operation. Customization and Configuration: The solution is highly configurable, allowing businesses to easily customize agent workflows, system interactions, and introduce new domain areas as needed. Advanced Technology Integration: Utilizing the latest real-time voice capability API from OpenAI and robust open-source SLM models, the accelerator is built on cutting-edge technology. Error Handling and Recovery: It incorporates error handling and recovery techniques to maintain conversation memory, ensuring a seamless customer experience even in the event of disruptions.   Challenges in Building AI for Customer Service While the benefits are clear, building AI for customer service presents several challenges: Human-Like Communication: To truly mimic a human agent, AI must communicate with all the nuances of human interaction while adhering to customer support processes and guidelines. Current chatbot technologies are often limited to predefined flows and simple intent detection. Integration with External Systems: Performing business transactions often requires complex interactions with external systems. This can involve multiple technical operations to complete a single business task. Maintaining Context: Keeping track of conversation context across multiple interactions is crucial, especially when technical disruptions occur. This challenge is also present in human interactions, where customers may lose context if transferred between agents. Complex Business Processes: Real-world customer service scenarios can be complex, requiring deep domain expertise that AI must emulate to be effective. Wide Coverage Needs: Customer service must encompass a broad range of products and business domains, each with its own complexities and process flows, necessitating a flexible and adaptable AI solution. By addressing these challenges, our AI-powered solution accelerator is poised to transform customer service, delivering efficient, scalable, and intelligent support that meets the demands of today's consumers. As technology continues to advance, businesses that embrace AI-driven customer service will be well-positioned to enhance customer satisfaction and drive success.   Key Solution Designs in the Accelerator There are a number of key design innovations developed in this solution accelerator to overcome the challenges described above. Most of these design features are shared between both the text and voice modalities, but there are some aspects that are unique to each modality.  Common Elements: Multi-Domain Agent Framework: To achieve a multi-domain agent solution, we have designed two patterns (one per modality) to orchestrate multiple individual agents—one for the hotel domain and another for the flight domain—so they can work seamlessly together. From the customer's perspective, these agents appear as a single customer service entity. Each domain agent is defined by a profile, which includes the system prompt and other agent specific data, and their associated tools, which are used to interact with source systems. This design means you can easily adapt the solution to your own use cases by replacing the sample agent profiles and tools with your own.  Stateful & Memory: State and memory are maintained across interactions, ensuring a coherent user experience both during agent transfers and in the event of connectivity issues. The solution provides an integration with Azure Redis to durably save session state along with an option for local in-memory storage for development.   Process Flow Definition: Clearly defined process flows guide the agent's actions, ensuring consistency and adherence to guidelines. These flows are defined in the agent profiles and their tools and are completely customizable to model your own workflows.  Source System Interactivity (Tool Calls): The ability to call external tools and systems is integrated into the service, allowing for seamless execution of complex tasks.  Headless Service: Operates independently of specific user interfaces, allowing flexibility in deployment.  Text Agent Specifics: Domain Agent Orchestration: To enable a seamless customer experience when interacting with multiple domain agents we’ve implemented a robust process to handle agent transfers. Underlying the individual agents is the Agent Runner, which manages the transfer process. Each agent is equipped with a `get_help` tool, which is called when the agent detects the conversation topic has moved out of its domain. When the agent calls for help the Agent Runner takes over to route the conversation to the appropriate agent.   To transfer the conversation, first the Agent Runner classifies the intent of the new topic by comparing the user’s request against the available agent’s domain descriptions. The classifier returns the intended agent’s name, which the Agent Runner then checks for validity and ensures that it differs from the current agent. This process repeats up to three times and if a valid agent is still not identified then the default agent is assigned to handle the user’s request. Finally, once a valid agent is identified, the Agent Runner assigns it as the active agent and supplies the conversation history to the new agent to ensure context is preserved through the transfer.  History Management: Includes capabilities for limiting and restoring conversation history as needed to stay within the context window limits of the model. Specifically, the solution provides the `clean_up_history` function to limit the conversation history only to user questions and agent responses, reducing the clutter of tool calls. Additionally, the function `reset_history_to_last_question` is provided to restrict the history to the last user question. These functions can be used to effectively manage the size of the history while maintaining appropriate context.  Voice Agent Specifics: Realtime API Capabilities: The voice modality, which is enabled by the GPT-4o Realtime API for speech and audio (Preview), unlocks exciting new possibilities for customer support scenarios. The model provides a number of features to address the challenges of using AI voice for customer service in real time.  Session State Lifecycle: Manages the state of voice sessions throughout their lifecycle through the WebSocket connection.  Voice Streaming Handling: Efficiently processes live voice streams for real-time interaction.  Interruption Handling: Capable of managing interruptions seamlessly, maintaining conversation flow. Tool Calls: Enables integrations with external systems for task execution during voice interactions.  Transcription Handling: Accurately transcribes voice interactions to enable conversation history tracking.  Session Management: Solution maintains session history using WebSocket sessions, the management of chat history is handled to ensure seamless interaction and continuity. Each session uniquely identifies and associated with a specific client, allowing aggregation of messages exchanged during that session.   It ensures that the conversation can be resumed or referenced as needed within the same session.   Decoupled Architecture: Our architecture is an extension of the pattern developed in VoiceRAG, which introduced a simple decoupled architecture for implementing RAG with the Azure OpenAI gpt-4o-realtime-preview model. We leveraged that pattern to provide multi-modal agentic capabilities for customer service scenarios where the voice-to-voice capabilities show huge value in improved customer experiences through highly personalized and responsive engagements. As in the original VoiceRAG pattern, the front-end client is decoupled from the middle tier which handles all interactions with the real-time model. This provides the benefits including:  Easy compatibility with any client that can work with Azure OpenAI API.  Enhanced security by preventing the client from accessing the model directly along with any configuration and credentials.  Domain Agent Orchestration (Taming the chatter): In our experience with the real-time API we discovered that a different approach was needed for agent orchestration from the text agent, where the agent itself was able to detect changes in the conversation topic and raise a request for assistance with a tool call. The real-time model is just too chatty to reliably respect that check and often prefers to respond rather than delegate via tool call. To mitigate this tendency, we’ve introduced an asynchronous intent monitoring process to identify when topic changes occur and assign the new agent before the existing agent can respond.   To enable this intent detection process, we leveraged the real-time API’s capability to provide transcriptions of the input user audio with the `input_audio_transcription` parameter. Once the transcription of the user’s request is returned by the model, we append it to the full conversation history, which is then sent to the intent detection model to determine which domain agent should be assigned to respond.   This intent detection process is powered by a low latency small language model (SLM). We’re presently using a fine-tuned Mistral-7B model for this task but are testing even lower latency SLMs like Phi-4. The SLM has been fine-tuned on a large, generated dataset of conversations representative of the domains and includes transitions between them to ensure these can be accurately detected by the model. When this model receives the conversation transcript it classifies the intent of the most recent user request and returns the name of the appropriate domain agent.   If the intended agent doesn’t match the currently assigned agent, then a transfer is initiated. The application resets the session with the real-time API using the profile and tools of the new agent. It also transfers the full conversation history by sending all prior messages as ` conversation.item.create` items to maintain conversation context with the customer.    History Management: The middle tier includes features to limit conversation history to a predefined limit to keep conversations within the context window limits.    These key design elements ensure that the solution accelerator delivers a robust, scalable, and versatile customer service AI capable of handling a wide range of interactions across different modalities and domains.     See how AI can enhance your customer service experience today  This solution is available on GitHub today to explore adding multi-modal agent capabilities to your customer service use cases: microsoft/multi-modal-customer-service-agent. Begin by exploring the travel agent use cases included in the repo sample and easily replace the agent profiles and capabilities with your own personas and systems to create a multi-modal customer service agentic system for your business.     

Overview A couple of weeks ago, Hesham and Hiten attended an internal Global Blackbelt summit in Redmond. Unfortunately, we encountered bad weather due to a “Bomb Cyclone”. Consequently, many people within the Washington State area were left without essential services such as electricity and running water. The Microsoft Campus was converted into a temporary relief center, leading to the rescheduling of our sessions for the day. Fortunately, we were staying at a hotel equipped with a backup generator, running water, and heating. We borrowed a whiteboard and spent the next few hours discussing the migration from 3 rd party solutions and mainly different query-based languages to Microsoft Sentinel – Defender XDR and KQL (Kusto Query Language) SIEM migrations are always a challenging process that requires meticulous planning and a thorough understanding of the existing setup in the legacy SIEM and what needs to be migrated to the modern SIEM. Our primary discussion point was how we can efficiently assist organizations in translating detection rules from query-based languages like AQL (Ariel Query Languages) to KQL (Kusto Query Language) and to help organizations mainly SOC teams to convert their YARA rules, STIX II and OpenIOC intel to KQL format   We identified two options: Convert AQL to KQL Convert YARA to KQL Convert STIX II and OpenIOC intel to KQL Both options require specialized skills and can be cumbersome to complete. Firstly, the user would need to understand precisely how AQL maps to KQL and second option require through knowledge of the syntax used in YARA generation. Both of these are niche skills.   One thing to point out here that we are not experts in query language like AQL and YARA Rules, we know how to export the detection rules in CVE format, our combined knowledge of AQL is also limited and would be considered basic at best.   We quickly realized that AQL is similar to SQL and consequently KQL. We have the following AQL Code: Select sourceip, destinationip, "Process Name" FROM events WHERE "Process Name" IMATCHES '.*atbroker\.exe.*|.*bash\.exe.*|.*bitsadmin\.exe.*|.*certutil\.exe.*|.*cmdkey\.exe.*|.*cmstp\.exe.*|.*control\.exe.*|.*csc\.exe.*|.*cscript\.exe.*|.*dfsvc\.exe.*|.*diskshadow\.exe.*|.*dnscmd\.exe.*|.*esentutl\.exe.*|.*eventvwr\.exe.*|.*expand\.exe.*|.*extexport\.exe.*|.*extrac32\.exe.*|.*findstr\.exe.*|.*forfiles\.exe.*|.*ftp\.exe.*|.*gpscript\.exe.*|.*hh\.exe.*|.*ie4uinit\.exe.*|.*ieexec\.exe.*|.*infdefaultinstall\.exe.*|.*installutil\.exe.*|.*makecab\.exe.*|.*reg\.exe.*|.*print\.exe.*|.*presentationhost\.exe.*|.*pcwrun\.exe.*|.*pcalua\.exe.*|.*odbcconf\.exe.*|.*msiexec\.exe.*|.*mshta\.exe.*|.*msdt\.exe.*|.*msconfig\.exe.*|.*msbuild\.exe.*|.*mmc\.exe.*|.*microsoft.workflow.compiler\.exe.*|.*mavinject\.exe.*|.*vsjitdebugger\.exe.*|.*tracker\.exe.*|.*te\.exe.*|.*sqltoolsps\.exe.*|.*sqlps\.exe.*|.*sqldumper\.exe.*|.*rcsi\.exe.*|.*msxsl\.exe.*|.*msdeploy\.exe.*|.*mftrace\.exe.*|.*dxcap\.exe.*|.*dnx\.exe.*|.*csi\.exe.*|.*cdb\.exe.*|.*bginfo\.exe.*|.*appvlp\.exe.*|.*xwizard\.exe.*|.*wsreset\.exe.*|.*wscript\.exe.*|.*wmic\.exe.*|.*wab\.exe.*|.*verclsid\.exe.*|.*syncappvpublishingserver\.exe.*|.*scriptrunner\.exe.*|.*schtasks\.exe.*|.*sc\.exe.*|.*runscripthelper\.exe.*|.*runonce\.exe.*|.*rundll32\.exe.*|.*rpcping\.exe.*|.*replace\.exe.*|.*regsvr32\.exe.*|.*regsvcs\.exe.*|.*register-cimprovider\.exe.*|.*regedit\.exe.*|.*regasm\.exe.*|' GROUP BY "Process Name",sourceip LAST 3 DAYS   This query aims to track the use of specific executables for administrative or potentially malicious activities. By grouping results by process name and source IP, it helps detect patterns or anomalies that may indicate security incidents or policy violations.   Using Microsoft Security Copilot What if we could get Security Copilot to firstly explain the code and then convert it to Kusto? We used the following prompt:   /askGPT I am planning a SIEM migration from AQL query based to Microsoft Sentinel KQL. As AQL expert can you create a detailed summary that explains the following AQL query in square brackets [ Select sourceip, destinationip, "Process Name" FROM events WHERE "Process Name" IMATCHES '.*atbroker\.exe.*|.*bash\.exe.*|.*bitsadmin\.exe.*|.*certutil\.exe.*|.*cmdkey\.exe.*|.*cmstp\.exe.*|.*control\.exe.*|.*csc\.exe.*|.*cscript\.exe.*|.*dfsvc\.exe.*|.*diskshadow\.exe.*|.*dnscmd\.exe.*|.*esentutl\.exe.*|.*eventvwr\.exe.*|.*expand\.exe.*|.*extexport\.exe.*|.*extrac32\.exe.*|.*findstr\.exe.*|.*forfiles\.exe.*|.*ftp\.exe.*|.*gpscript\.exe.*|.*hh\.exe.*|.*ie4uinit\.exe.*|.*ieexec\.exe.*|.*infdefaultinstall\.exe.*|.*installutil\.exe.*|.*makecab\.exe.*|.*reg\.exe.*|.*print\.exe.*|.*presentationhost\.exe.*|.*pcwrun\.exe.*|.*pcalua\.exe.*|.*odbcconf\.exe.*|.*msiexec\.exe.*|.*mshta\.exe.*|.*msdt\.exe.*|.*msconfig\.exe.*|.*msbuild\.exe.*|.*mmc\.exe.*|.*microsoft.workflow.compiler\.exe.*|.*mavinject\.exe.*|.*vsjitdebugger\.exe.*|.*tracker\.exe.*|.*te\.exe.*|.*sqltoolsps\.exe.*|.*sqlps\.exe.*|.*sqldumper\.exe.*|.*rcsi\.exe.*|.*msxsl\.exe.*|.*msdeploy\.exe.*|.*mftrace\.exe.*|.*dxcap\.exe.*|.*dnx\.exe.*|.*csi\.exe.*|.*cdb\.exe.*|.*bginfo\.exe.*|.*appvlp\.exe.*|.*xwizard\.exe.*|.*wsreset\.exe.*|.*wscript\.exe.*|.*wmic\.exe.*|.*wab\.exe.*|.*verclsid\.exe.*|.*syncappvpublishingserver\.exe.*|.*scriptrunner\.exe.*|.*schtasks\.exe.*|.*sc\.exe.*|.*runscripthelper\.exe.*|.*runonce\.exe.*|.*rundll32\.exe.*|.*rpcping\.exe.*|.*replace\.exe.*|.*regsvr32\.exe.*|.*regsvcs\.exe.*|.*register-cimprovider\.exe.*|.*regedit\.exe.*|.*regasm\.exe.*|' GROUP BY "Process Name",sourceip LAST 3 DAYS]   Below is the OUTPUT Detailed Summary of AQL Query Breakdown Query Overview The provided AQL query is designed to search for specific process names within event logs and return the source and destination IP addresses associated with those processes. Here's a detailed breakdown of the query: AQL Query Breakdown Select Clause: sourceip: The source IP address from which the event originated. destinationip: The destination IP address to which the event was directed. "Process Name": The name of the process involved in the event. From Clause: events: The dataset or table from which the query is retrieving data. Where Clause: "Process Name" IMATCHES '.*atbroker\.exe.*|.*bash\.exe.*|.*bitsadmin\.exe.*|.*certutil\.exe.*|.*cmdkey\.exe.*|.*cmstp\.exe.*|.*control\.exe.*|.*csc\.exe.*|.*cscript\.exe.*|.*dfsvc\.exe.*|.*diskshadow\.exe.*|.*dnscmd\.exe.*|.*esentutl\.exe.*|.*eventvwr\.exe.*|.*expand\.exe.*|.*extexport\.exe.*|.*extrac32\.exe.*|.*findstr\.exe.*|.*forfiles\.exe.*|.*ftp\.exe.*|.*gpscript\.exe.*|.*hh\.exe.*|.*ie4uinit\.exe.*|.*ieexec\.exe.*|.*infdefaultinstall\.exe.*|.*installutil\.exe.*|.*makecab\.exe.*|.*reg\.exe.*|.*print\.exe.*|.*presentationhost\.exe.*|.*pcwrun\.exe.*|.*pcalua\.exe.*|.*odbcconf\.exe.*|.*msiexec\.exe.*|.*mshta\.exe.*|.*msdt\.exe.*|.*msconfig\.exe.*|.*msbuild\.exe.*|.*mmc\.exe.*|.*microsoft.workflow.compiler\.exe.*|.*mavinject\.exe.*|.*vsjitdebugger\.exe.*|.*tracker\.exe.*|.*te\.exe.*|.*sqltoolsps\.exe.*|.*sqlps\.exe.*|.*sqldumper\.exe.*|.*rcsi\.exe.*|.*msxsl\.exe.*|.*msdeploy\.exe.*|.*mftrace\.exe.*|.*dxcap\.exe.*|.*dnx\.exe.*|.*csi\.exe.*|.*cdb\.exe.*|.*bginfo\.exe.*|.*appvlp\.exe.*|.*xwizard\.exe.*|.*wsreset\.exe.*|.*wscript\.exe.*|.*wmic\.exe.*|.*wab\.exe.*|.*verclsid\.exe.*|.*syncappvpublishingserver\.exe.*|.*scriptrunner\.exe.*|.*schtasks\.exe.*|.*sc\.exe.*|.*runscripthelper\.exe.*|.*runonce\.exe.*|.*rundll32\.exe.*|.*rpcping\.exe.*|.*replace\.exe.*|.*regsvr32\.exe.*|.*regsvcs\.exe.*|.*register-cimprovider\.exe.*|.*regedit\.exe.*|.*regasm\.exe.*': This condition filters events to include only those where the process name matches any of the specified patterns. The IMATCHES operator is used for case-insensitive regular expression matching. Group By Clause: "Process Name", sourceip: The results are grouped by the process name and source IP address. Time Range: LAST 3 DAYS: The query is limited to events that occurred within the last three days. Summary This AQL query is designed to identify and group events based on specific process names that are often associated with suspicious or malicious activity. By focusing on these processes, the query helps in monitoring and investigating potential security incidents. The results provide insights into the source and destination IP addresses involved, which can be crucial for further analysis and response. The query returns the process name, source IP, and destination IP for events matching the specified process names, grouped by process name and source IP, within the last three days.   This next step to see if Security Copilot can help convert the AQL to something that is mapped to Microsoft Sentinel Kusto Query Language: Can you analyze the AQL query above and map it to KQL, it is important to ensure that the mapping directly correlates to the schema in KQL. Can you also show the output of the query in a table. Present the mappings in a table:   KQL Query and Output KQL Query Mapping DeviceProcessEvents | where TimeGenerated >= ago(3d) | where InitiatingProcessFileName matches regex @'(?i).*atbroker\.exe.*|.*bash\.exe.*|.*bitsadmin\.exe.*|.*certutil\.exe.*|.*cmdkey\.exe.*|.*cmstp\.exe.*|.*control\.exe.*|.*csc\.exe.*|.*cscript\.exe.*|.*dfsvc\.exe.*|.*diskshadow\.exe.*|.*dnscmd\.exe.*|.*esentutl\.exe.*|.*eventvwr\.exe.*|.*expand\.exe.*|.*extexport\.exe.*|.*extrac32\.exe.*|.*findstr\.exe.*|.*forfiles\.exe.*|.*ftp\.exe.*|.*gpscript\.exe.*|.*hh\.exe.*|.*ie4uinit\.exe.*|.*ieexec\.exe.*|.*infdefaultinstall\.exe.*|.*installutil\.exe.*|.*makecab\.exe.*|.*reg\.exe.*|.*print\.exe.*|.*presentationhost\.exe.*|.*pcwrun\.exe.*|.*pcalua\.exe.*|.*odbcconf\.exe.*|.*msiexec\.exe.*|.*mshta\.exe.*|.*msdt\.exe.*|.*msconfig\.exe.*|.*msbuild\.exe.*|.*mmc\.exe.*|.*microsoft.workflow.compiler\.exe.*|.*mavinject\.exe.*|.*vsjitdebugger\.exe.*|.*tracker\.exe.*|.*te\.exe.*|.*sqltoolsps\.exe.*|.*sqlps\.exe.*|.*sqldumper\.exe.*|.*rcsi\.exe.*|.*msxsl\.exe.*|.*msdeploy\.exe.*|.*mftrace\.exe.*|.*dxcap\.exe.*|.*dnx\.exe.*|.*csi\.exe.*|.*cdb\.exe.*|.*bginfo\.exe.*|.*appvlp\.exe.*|.*xwizard\.exe.*|.*wsreset\.exe.*|.*wscript\.exe.*|.*wmic\.exe.*|.*wab\.exe.*|.*verclsid\.exe.*|.*syncappvpublishingserver\.exe.*|.*scriptrunner\.exe.*|.*schtasks\.exe.*|.*sc\.exe.*|.*runscripthelper\.exe.*|.*runonce\.exe.*|.*rundll32\.exe.*|.*rpcping\.exe.*|.*replace\.exe.*|.*regsvr32\.exe.*|.*regsvcs\.exe.*|.*register-cimprovider\.exe.*|.*regedit\.exe.*|.*regasm\.exe.*' | summarize by InitiatingProcessFileName   Query Output InitiatingProcessFileName csc.exe gpupdate.exe microsoftedgeupdate.exe msiexec.exe schtasks.exe sc.exe cscript.exe rundll32.exe makecab.exe route.exe Row Counts Total Rows: 10 Displayed Rows: 10 Product Name Product: Sentinel   You can download the following from Microsoft Security Copilot official Github repo, thanks Sean_Wasonga for supporting the automated deployment package. KQL Migrator Azure Logic App Code file : Click here o   Make sure to replace the following with your tenant's correct information: <subscription-ID> <ResourceGroup-Name> Sample AQL queries / YARA rules: Click here   The extended team highlighted the importance of considering YARA rules for uniformity. Given that expertise in YARA rule generation was lacking, Security Copilot as again consulted:     /askGPT Can you explain what a YAYA rule is with an example A YARA rule is a type of rule used in cybersecurity to identify and classify malware samples. YARA, which stands for "Yet Another Recursive Acronym," is a tool aimed at helping malware researchers identify and classify malware. YARA rules are written in a specific syntax and are used to describe patterns that can be found in files or processes. Example of a YARA Rule Here is a simple example of a YARA rule that detects a file containing the string "malicious": rule ExampleRule {     meta:         description = "This rule detects files containing the string 'malicious'"         author = "Security Researcher"         date = "2024-12-09"       strings:         $malicious_string = "malicious"       condition:         $malicious_string }   So we have a base understand of what a YARA rule is and its typical format. The step is can we use Security Copilot to convert the YARA into a KQL Rule:   We used the following prompt:   As a YARA rule expert, can you take the above KQL query generated and create a KQL rule? rule Suspicious_Process_Execution { meta: description = "Detects execution of potentially suspicious processes" author = "Security Analyst" date = "2023-10-01" strings: $atbroker = /(?i).*atbroker\.exe.*/ $bash = /(?i).*bash\.exe.*/ $bitsadmin = /(?i).*bitsadmin\.exe.*/ $certutil = /(?i).*certutil\.exe.*/ $cmdkey = /(?i).*cmdkey\.exe.*/ $cmstp = /(?i).*cmstp\.exe.*/ $control = /(?i).*control\.exe.*/ $csc = /(?i).*csc\.exe.*/ $cscript = /(?i).*cscript\.exe.*/ $dfsvc = /(?i).*dfsvc\.exe.*/ $diskshadow = /(?i).*diskshadow\.exe.*/ $dnscmd = /(?i).*dnscmd\.exe.*/ $esentutl = /(?i).*esentutl\.exe.*/ $eventvwr = /(?i).*eventvwr\.exe.*/ $expand = /(?i).*expand\.exe.*/ $extexport = /(?i).*extexport\.exe.*/ $extrac32 = /(?i).*extrac32\.exe.*/ $findstr = /(?i).*findstr\.exe.*/ $forfiles = /(?i).*forfiles\.exe.*/ $ftp = /(?i).*ftp\.exe.*/ $gpscript = /(?i).*gpscript\.exe.*/ $hh = /(?i).*hh\.exe.*/ $ie4uinit = /(?i).*ie4uinit\.exe.*/ $ieexec = /(?i).*ieexec\.exe.*/ $infdefaultinstall = /(?i).*infdefaultinstall\.exe.*/ $installutil = /(?i).*installutil\.exe.*/ $makecab = /(?i).*makecab\.exe.*/ $reg = /(?i).*reg\.exe.*/ $print = /(?i).*print\.exe.*/ $presentationhost = /(?i).*presentationhost\.exe.*/ $pcwrun = /(?i).*pcwrun\.exe.*/ $pcalua = /(?i).*pcalua\.exe.*/ $odbcconf = /(?i).*odbcconf\.exe.*/ $msiexec = /(?i).*msiexec\.exe.*/ $mshta = /(?i).*mshta\.exe.*/ $msdt = /(?i).*msdt\.exe.*/ $msconfig = /(?i).*msconfig\.exe.*/ $msbuild = /(?i).*msbuild\.exe.*/ $mmc = /(?i).*mmc\.exe.*/ $microsoft_workflow_compiler = /(?i).*microsoft\.workflow\.compiler\.exe.*/ $mavinject = /(?i).*mavinject\.exe.*/ $vsjitdebugger = /(?i).*vsjitdebugger\.exe.*/ $tracker = /(?i).*tracker\.exe.*/ $te = /(?i).*te\.exe.*/ $sqltoolsps = /(?i).*sqltoolsps\.exe.*/ $sqlps = /(?i).*sqlps\.exe.*/ $sqldumper = /(?i).*sqldumper\.exe.*/ $rcsi = /(?i).*rcsi\.exe.*/ $msxsl = /(?i).*msxsl\.exe.*/ $msdeploy = /(?i).*msdeploy\.exe.*/ $mftrace = /(?i).*mftrace\.exe.*/ $dxcap = /(?i).*dxcap\.exe.*/ $dnx = /(?i).*dnx\.exe.*/ $csi = /(?i).*csi\.exe.*/ $cdb = /(?i).*cdb\.exe.*/ $bginfo = /(?i).*bginfo\.exe.*/ $appvlp = /(?i).*appvlp\.exe.*/ $xwizard = /(?i).*xwizard\.exe.*/ $wsreset = /(?i).*wsreset\.exe.*/ $wscript = /(?i).*wscript\.exe.*/ $wmic = /(?i).*wmic\.exe.*/ $wab = /(?i).*wab\.exe.*/ $verclsid = /(?i).*verclsid\.exe.*/ $syncappvpublishingserver = /(?i).*syncappvpublishingserver\.exe.*/ $scriptrunner = /(?i).*scriptrunner\.exe.*/ $schtasks = /(?i).*schtasks\.exe.*/ $sc = /(?i).*sc\.exe.*/ $runscripthelper = /(?i).*runscripthelper\.exe.*/ $runonce = /(?i).*runonce\.exe.*/ $rundll32 = /(?i).*rundll32\.exe.*/ $rpcping = /(?i).*rpcping\.exe.*/ $replace = /(?i).*replace\.exe.*/ $regsvr32 = /(?i).*regsvr32\.exe.*/ $regsvcs = /(?i).*regsvcs\.exe.*/ $register_cimprovider = /(?i).*register-cimprovider\.exe.*/ $regedit = /(?i).*regedit\.exe.*/ $regasm = /(?i).*regasm\.exe.*/ condition: any of ($atbroker, $bash, $bitsadmin, $certutil, $cmdkey, $cmstp, $control, $csc, $cscript, $dfsvc, $diskshadow, $dnscmd, $esentutl, $eventvwr, $expand, $extexport, $extrac32, $findstr, $forfiles, $ftp, $gpscript, $hh, $ie4uinit, $ieexec, $infdefaultinstall, $installutil, $makecab, $reg, $print, $presentationhost, $pcwrun, $pcalua, $odbcconf, $msiexec, $mshta, $msdt, $msconfig, $msbuild, $mmc, $microsoft_workflow_compiler, $mavinject, $vsjitdebugger, $tracker, $te, $sqltoolsps, $sqlps, $sqldumper, $rcsi, $msxsl, $msdeploy, $mftrace, $dxcap, $dnx, $csi, $cdb, $bginfo, $appvlp, $xwizard, $wsreset, $wscript, $wmic, $wab, $verclsid, $syncappvpublishingserver, $scriptrunner, $schtasks, $sc, $runscripthelper, $runonce, $rundll32, $rpcping, $replace, $regsvr32, $regsvcs, $register_cimprovider, $regedit, $regasm) } KQL Rule Based on YARA Rule   DeviceProcessEvents | where Timestamp >= ago(3d) | where InitiatingProcessFileName matches regex @'(?i).*atbroker\.exe.*|.*bash\.exe.*|.*bitsadmin\.exe.*|.*certutil\.exe.*|.*cmdkey\.exe.*|.*cmstp\.exe.*|.*control\.exe.*|.*csc\.exe.*|.*cscript\.exe.*|.*dfsvc\.exe.*|.*diskshadow\.exe.*|.*dnscmd\.exe.*|.*esentutl\.exe.*|.*eventvwr\.exe.*|.*expand\.exe.*|.*extexport\.exe.*|.*extrac32\.exe.*|.*findstr\.exe.*|.*forfiles\.exe.*|.*ftp\.exe.*|.*gpscript\.exe.*|.*hh\.exe.*|.*ieuinit\.exe.*|.*ieexec\.exe.*|.*infdefaultinstall\.exe.*|.*installutil\.exe.*|.*makecab\.exe.*|.*reg\.exe.*|.*print\.exe.*|.*presentationhost\.exe.*|.*pcwrun\.exe.*|.*pcalua\.exe.*|.*odbcconf\.exe.*|.*msiexec\.exe.*|.*mshta\.exe.*|.*msdt\.exe.*|.*msconfig\.exe.*|.*msbuild\.exe.*|.*mmc\.exe.*|.*microsoft.workflow.compiler\.exe.*|.*mavinject\.exe.*|.*vsjitdebugger\.exe.*|.*tracker\.exe.*|.*te\.exe.*|.*sqltoolsps\.exe.*|.*sqlps\.exe.*|.*sqldumper\.exe.*|.*rcsi\.exe.*|.*msxsl\.exe.*|.*msdeploy\.exe.*|.*mftrace\.exe.*|.*dxcap\.exe.*|.*dnx\.exe.*|.*csi\.exe.*|.*cdb\.exe.*|.*bginfo\.exe.*|.*appvlp\.exe.*|.*xwizard\.exe.*|.*wsreset\.exe.*|.*wscript\.exe.*|.*wmic\.exe.*|.*wab\.exe.*|.*verclsid\.exe.*|.*syncappvpublishingserver\.exe.*|.*scriptrunner\.exe.*|.*schtasks\.exe.*|.*sc\.exe.*|.*runscripthelper\.exe.*|.*runonce\.exe.*|.*rundll32\.exe.*|.*rpcping\.exe.*|.*replace\.exe.*|.*regsvr32\.exe.*|.*regsvcs\.exe.*|.*register-cimprovider\.exe.*|.*regedit\.exe.*|.*regasm\.exe.*' | summarize by InitiatingProcessFileName   We now have a method to efficiently convert a detection rule from a legacy SIEM to one that is compatible with Microsoft Sentinel . The next step is understanding how a user can implement this in Sentinel. It is important to consider that the user may need to acquaint themselves with the new SIEM or hire a contractor. Since a contractor wasn’t available, given the circumstances, we decided to ask Security Copilot: Can you create a change control document that shows the steps required to create the detection rule for the KQL query generated above in sentinel. Also include the required level of access to make such a change. Ensure rollback is included. Change Control Document: Creating a Detection Rule in Microsoft Sentinel Change Request Details Change Request ID: [To be assigned] Change Title: Create Detection Rule for Suspicious Process Execution Change Description: Implement a detection rule in Microsoft Sentinel based on the provided KQL query to monitor for suspicious process executions. Steps to Implement the Change Log in to Azure Portal: Navigate to the Azure portal (https://portal.azure.com). Ensure you have the necessary permissions to access Microsoft Sentinel. Access Microsoft Sentinel: In the Azure portal, search for "Microsoft Sentinel" and select the appropriate workspace. Create a New Analytics Rule: Go to the "Analytics" section in the Microsoft Sentinel workspace. Click on "Create" and select "Scheduled query rule." Define Rule Details: Name: Suspicious Process Execution Detection Description: Detects execution of potentially suspicious processes based on the provided KQL query. Severity: High Tactics: Execution Set Rule Logic: Query: Copy and paste the following KQL query: DeviceProcessEvents | where TimeGenerated >= ago(3d) | where InitiatingProcessFileName matches regex @'(?i).*atbroker\.exe.*|.*bash\.exe.*|.*bitsadmin\.exe.*|.*certutil\.exe.*|.*cmdkey\.exe.*|.*cmstp\.exe.*|.*control\.exe.*|.*csc\.exe.*|.*cscript\.exe.*|.*dfsvc\.exe.*|.*diskshadow\.exe.*|.*dnscmd\.exe.*|.*esentutl\.exe.*|.*eventvwr\.exe.*|.*expand\.exe.*|.*extexport\.exe.*|.*extrac32\.exe.*|.*findstr\.exe.*|.*forfiles\.exe.*|.*ftp\.exe.*|.*gpscript\.exe.*|.*hh\.exe.*|.*ie4uinit\.exe.*|.*ieexec\.exe.*|.*infdefaultinstall\.exe.*|.*installutil\.exe.*|.*makecab\.exe.*|.*reg\.exe.*|.*print\.exe.*|.*presentationhost\.exe.*|.*pcwrun\.exe.*|.*pcalua\.exe.*|.*odbcconf\.exe.*|.*msiexec\.exe.*|.*mshta\.exe.*|.*msdt\.exe.*|.*msconfig\.exe.*|.*msbuild\.exe.*|.*mmc\.exe.*|.*microsoft.workflow.compiler\.exe.*|.*mavinject\.exe.*|.*vsjitdebugger\.exe.*|.*tracker\.exe.*|.*te\.exe.*|.*sqltoolsps\.exe.*|.*sqlps\.exe.*|.*sqldumper\.exe.*|.*rcsi\.exe.*|.*msxsl\.exe.*|.*msdeploy\.exe.*|.*mftrace\.exe.*|.*dxcap\.exe.*|.*dnx\.exe.*|.*csi\.exe.*|.*cdb\.exe.*|.*bginfo\.exe.*|.*appvlp\.exe.*|.*xwizard\.exe.*|.*wsreset\.exe.*|.*wscript\.exe.*|.*wmic\.exe.*|.*wab\.exe.*|.*verclsid\.exe.*|.*syncappvpublishingserver\.exe.*|.*scriptrunner\.exe.*|.*schtasks\.exe.*|.*sc\.exe.*|.*runscripthelper\.exe.*|.*runonce\.exe.*|.*rundll32\.exe.*|.*rpcping\.exe.*|.*replace\.exe.*|.*regsvr32\.exe.*|.*regsvcs\.exe.*|.*register-cimprovider\.exe.*|.*regedit\.exe.*|.*regasm\.exe.*' | summarize by InitiatingProcessFileName Query Frequency: 5 minutes Query Period: 3 days Set Alert Details: Alert Rule Name: Suspicious Process Execution Detected Description: Alerts when a suspicious process execution is detected. Severity: High Tactics: Execution Configure Automated Response (Optional): Add any playbooks or automated responses if required. Review and Create: Review the rule configuration. Click "Create" to save and activate the rule. Required Access Level Role: Security Administrator or Sentinel Contributor Permissions: Read and write access to Microsoft Sentinel Ability to create and manage analytics rules Rollback Plan Log in to Azure Portal: Navigate to the Azure portal (https://portal.azure.com). Access Microsoft Sentinel: In the Azure portal, search for "Microsoft Sentinel" and select the appropriate workspace. Delete the Analytics Rule: Go to the "Analytics" section in the Microsoft Sentinel workspace. Locate the "Suspicious Process Execution Detection" rule. Select the rule and click "Delete." Verify Deletion: Ensure the rule is no longer listed in the "Analytics" section. Document the Rollback: Update the change control document to reflect the rollback action. Approval Change Approver: [To be assigned] Approval Date: [To be assigned] Implementation Date Scheduled Date: [To be assigned] Implementation Time: [To be assigned] Rollback Date Scheduled Date: [To be assigned] Rollback Time: [To be assigned]   As you can see with the above methods, we have the option now to create a prompt book which can be manually run to efficiently convert AQL to either KQL or YARA.   But since we had more time to play with, we decide to see if we could do this task in an automated fashion using logic apps. We exported the AQL queries and imported them directly into Watchlists within Sentinel:   Clicking on view logs shows AQLPayLoad stored in the _GetWatchlist('AQL2KQL') table: Since we have this watch list now imported, it simply will not work as the syntax of AQL is not understood by Sentinel: We now have the option of getting Copilot to carry out the conversion via a Logic App so in theory we just have to upload the watch list and let the logic app do the conversation. Conclusion and Consideration So, we've explored how Security Copilot Azure Logic App connector and Promptbooks revolutionizes the integration of AI-driven security solutions into current workflows. This tool not only provides greater customization and smoother system integration but also access to a broader range of ready-made Azure logic security tools integration. We encourage you to experience the efficiency of the Security Copilot Azure Logic App connector in enhancing your security operations. Whilst investigating and understanding AQL/YARA we did not leave the copilot platform You can customize the Logic App by automatically creating the Microsoft Sentinel analytics rules via the API If Logic Apps are not suitable, consider adding a step to create control documentation within the promptbook. Authors Hesham Saad - Sr Cybersecurity Global Blackbelt Hiten Sharma – Sr Cybersecurity Global Blackbelt

The Microsoft Careers Portal processes 10 million job applications per year. For a tailored user experience, it uses Azure AI Search for features like hierarchical facets and dynamic filtering.  Faceted navigation and filtering are vital components in modern search applications, enhancing the ability to deliver precise, contextually relevant results. E-commerce websites often utilize filters to help users refine product searches, while more advanced applications, such as those powered by Azure AI Search, extend these capabilities to support features like geo-spatial filtering, hierarchical facets, and dynamic filtering for a tailored user experience.  This case study examines the use of Azure AI Search within the Microsoft Careers Portal, which processes roughly 10 million job applications annually. The study highlights the complexities of implementing multi-layered, interconnected filters and facets in an enterprise setting. By default, Azure AI Search provides counts for specified facets when a filter value is selected; however, additional customization ensures dynamic updates for filter and facet counts across unselected categories. This paper proposes an innovative approach for leveraging Azure AI Search’s existing capabilities to handle these complex requirements, offering a  scalable solution applicable  across diverse enterprise use cases.    1. Introduction  The Microsoft Careers Portal integrates several first-party Microsoft products to deliver a seamless and user-centric experience for both internal employees and external job seekers. The portal provides various filters, including Experience, Work-Site Preference, Profession, and Employment Type, which are tailored based on user profiles to streamline the search for relevant job opportunities.    Built on Azure AI Search, the portal offers advanced search capabilities such as Boolean search, exact- match, fuzzy search and semantic ranking. These features enhance the user experience by ensuring that job listings are accurate and relevant. However, when users select multiple filters across categories, the complexity increases in maintaining accurate facet counts in real-time. Despite these challenges, Azure AI Search supports a robust faceted navigation experience, dynamically adjusting filter counts to reflect ongoing user selections with custom solution shared in this study.    2. Azure AI Search: Key Features Capabilities  2.1 Basic Concepts and features Azure AI Search provides a scalable, secure search platform capable of handling both traditional keyword and AI-augmented retrieval applications such as vector and hybrid search. The following are it’s key components:  Comprehensive Retrieval System: Supports full-text, hybrid, and vector search within an index, with field-level faceting enabled by setting fields as ”facetable.”  Advanced Query Syntax: Facilitates complex queries, including hybrid queries, fuzzy search, auto- complete, geo-search, and vector queries, enabling refined control over search functionality.  Semantic Relevance Tuning: Offers semantic ranking, document boosting via scoring profiles, vector weighting, and other runtime parameters for optimizing query behavior. Language Analyzers: An analyzer is a component of full-text search engine responsible for processing strings during indexing and query execution.  OData Filter Expressions: Provides granular control over filtering, with support for combining Boolean and full-text search expressions.   2.2 Filters and Faceted Navigation Azure AI Search, filters and facets provide users with a refined search experience:  Faceted Navigation: Enables users to interactively filter results, such as job type  or  location, through an intuitive UI.  Filterable Fields: These fields allow filtering operations, where fields marked as ”filterable” increase the index size. It’s recommended to disable ”filterable” for fields not used in filters to optimize performance. Example Request: Filtering for results where the BaseRate is less than 150 in a Rooms collection is illustrated in Listing 1 below: 1: POST Request for Filtering ‘BaseRate‘ in Rooms Faceted navigation is used for self-directed drilldown filtering on query results in a search app, where your application offers form controls for scoping search to groups of documents (for example, categories or brands), to support the experience.    2: Facets are specified on the query as request below   3: Faceted navigation structure is returned as below Text filters match string fields against literal strings that you provide in the filter. Unlike full-text search, there’s no lexical analysis or word-breaking for text filters, so comparisons are for exact matches only. For example: $filter=Category eq 'Resort and Spa', will only filter documents with text - 'Resort and Spa'.  Approaches for filtering on text:  search.in: A function that matches a field against a delimited list of strings. It is used where many raw text values need to be matched with a string field.  search.ismatch: A function that allows you to mix full-text search operations with strictly Boolean filter operations in the same filter expression. It is used where we want multiple search-filter combinations in one request.  $filter=field operator string: A user-defined expression composed of fields, operators, and values. It is used to find exact matches between a string field and a string value.    3. Customized Implementation for Microsoft Careers Portal 3.1 Career’s site requirement with Filter & Faceted navigation  The Microsoft Careers Portal required an approach to dynamically update filter and facet counts as users interacted with the search filters. Specifically, when a user selects a filter value within one category, Azure AI Search should update facet counts across all other unselected categories. This requirement ensures that users receive accurate results reflecting the available job listings, even as filter selections evolve.    For example, when a user selects ”Software Engineering” under the Profession filter, counts in related facets (such as Discipline and Work Site) are adjusted based on the available jobs in that profession. This behavior is visually demonstrated in Figure 1 below.    1: Faceted Navigation with Dynamic Filter Counts on Microsoft Careers   3.2 Solution  Approach The solution involves four categories of filters (A, B, C, and D). When a user selects values from Categories A and B, the system updates the facet counts across other categories as follows:  Primary Query Execution: The selected values within the same category are combined with OR, and across categories with AND, to generate an accurate search result set.  Updating Filter Values in Unselected Categories: Additional queries are executed for categories without selected values to retrieve updated counts. This iterative query approach ensures that unselected facets reflect the correct result counts. This approach allows the Microsoft Careers Portal to deliver a dynamic, real-time faceted navigation experience, keeping filter counts accurate and improving user satisfaction. & search queries triggered parallelly 3.3 Best Practices Learned While Implementation Custom analyzer on specific fields helps to enhance search for documents having matching keywords. For example, in job descriptions we have #hastags based keyword which are used with jobs posted during campaigns or some teams for boosting search. A custom-analyzer is invoked on a per-field basis and is recommended to use to cater dynamic search needs. Define scoring profiles cautiously: Prioritize Important Fields: Assign higher weights to fields that are more relevant to the search context. For example, the "Title" field has a higher weight compared to other fields, indicating its importance in search relevance. Use Freshness Boosts: Incorporate freshness boosts to prioritize recent content. This is particularly useful for time-sensitive data. Adjust the boost value and interpolation method based on the desired impact. For instance, a higher boost with linear interpolation is used for recency-sensitive profiles. Combine Multiple Scoring Functions: Use a combination of text weights and scoring functions to achieve a balanced relevance score. The functionAggregation method "sum" is used to aggregate the scores from different functions. Test and Iterate: Regularly test and refine scoring profiles based on search performance and user feedback. Adjust weights, boost values, and interpolation methods as needed to improve search relevance. 3.4 Performance Evaluation A service-side performance test was conducted in a production-cloned environment at Azure Test Runner to validate the implementation under high-load conditions, with the portal supporting approximately 50,000-60,000 searches daily. Our search-service app service triggered requests directly to Azure AI Search deployed instance. Performance results are shown below:    Request Per Second  Filters Count  Average  Latency(ms)  20 RPS  1  429  30 RPS  1  635  30 RPS  21  482  30 RPS  70  712    Performance was optimized with a replica count of 1-7 and a consistent partition count of 1, with Web App SKU - S1 App Service Plan and scale-out configuration between 1-3 instances on below:  (Average) CPU consumption > 70% (Average) Memory percentage > 80% (Average) HTTP Response time > 10s   3.5 Conclusion This case study demonstrates how Azure AI Search can effectively address complex requirements for faceted navigation in high-traffic, enterprise-level applications like Microsoft Careers. By enabling real- time, multi-layered filter updates, Azure AI Search not only meets but exceeds industry standards for search performance and relevance, reinforcing its position as a state-of-the-art solution for sophisticated search and retrieval needs. For developers and architects looking to implement similar capabilities, Azure AI Search provides a comprehensive platform capable of scaling to meet diverse business requirements.    Contributors: Prachi Nautiyal, Pradip Takate, Farzad Sunavala, Abhishek Mishra, Bipul Raman, Satya Vamsi Gadikoyila, Ashudeep Reshi