How do you connect machine learning with other software?

Building a machine learning model involves: Data collection: Gather diverse and high-quality data for training. Data preprocessing: Clean, transform, and organize data for effective training. Normalize, scale features, and handle missing values. Model selection: Choose the suitable algorithm and architecture. Training: Feed prepared data to the model for pattern identification through optimization. Evaluation: Assess model performance using a separate dataset (validation or test) to ensure generalization to new data. Deployment: Integrate the trained model into real-world applications through software systems or dedicated services/APIs. Monitoring and updating: Regularly monitor models to maintain performance as data changes.

When embarking on a project that involves building Machine Learning models, it is crucial to consider both functional and nonfunctional requirements. The process of acquiring data till finalizing the model has been thoroughly studied and explained. When integrating with IT systems, we must consider platform choice, regulatory requirements, and cost for ML model decisions. The cost of incremental maintenance of the system, risk, and mitigation approaches should be discussed and documented. Above all, establishing observability with the DataOps team ensures the ML system is well integrated with the IT solution.

Yes they can be a part of the solution but assuming they are the full solution sometimes leads to failures in real world scenarios. Firstly as pointed out, they need to be trained with a proper data pipeline, experiment tracking and deployment strategies that may even include compression and/or conversions. After getting a deployed model, the inference by it should be considered as a signal operating on a subset of the input data. This signal should be further refined, smoothed in real time, adjusted for edge cases and used in a meaningful way to solve the use case, all with an additional layer of logic that is independent of machine learning inference.

In my experience, data preparation is most important for successful deployment. Dirty data translates to poor predictions. For example, data that is poorly labeled, or with labels that are not relevant, will inevitably result in the AI learning something wrong or useless for the scope of the predictions (inference). For this reason, training data must be as clean as possible. And as much as possible. Poor data size also affects negatively accuracy of the inference.

The choice of deployment method depends on the specific use case and the trade-offs between simplicity, efficiency, scalability, and flexibility. For instance, in resource-constrained environments or for simple models, embedding might be the most practical choice. On the other hand, for large-scale, complex models that require frequent updates and scalability, an API or containerization could be more suitable. It is crucial to carefully assess the project's needs, the available resources, and the expertise of the development team to choose the most appropriate deployment strategy. It is also important to have monitoring and maintenance processes in place to ensure the models continue to perform effectively and meet the desired objectives.

When planning, it is important to gather the nonfunctional requirements (NFR) of the machine learning (ML) model. Additionally, it is necessary to consider the larger IT system/product's vision, as both factors will influence the deployment options. For instance, in a use case where an ML model is built to detect the tone, intent, and sentiment of customer chat conversations, some sample NFRs include: 1) The number of users per minute. 2) The number of messages to be processed per minute. 3) The expected system latency. 4) The fallback system to be used if the ML model is not available. Based on these requirements and NFRs, an API deployment model is recommended. The deployment model can then be designed to meet the specific NFRs.

Each option has its benefits and considerations, depending on the characteristics of your target software. Embedding the model into an application is fast but might lack flexibility for complex models. An API, while scalable, can introduce latency. Containerization, using Docker, creates robust, portable units, but demands higher technical skill. Python's "Flask" or "FastAPI" are ok for serving models as APIs, while Docker + Kubernetes is a common go-to for containerization.

Agree, but in defense of embedding a model, it should be kept in mind that this becomes one of the only option when data privacy is also something to care about. For example a face indentification application sending recorded data to some cloud server across countries may violate many data collection and usage laws, meaning the cloud servers in this case may process app logic but sensitive data processing by the model needs to happen on site.

Choosing the right communication method is crucial in ensuring the overall performance and user experience of the deployed machine learning model. Factors such as response time requirements, resource constraints, and the nature of the application will influence the decision. Additionally, the scalability and reliability of the chosen method should be considered, as a model might need to handle varying loads and be resilient to potential failures.

A fun metaphor to conceptualize this: Once your ML model is deployed, it's like a newly added actor on a theater stage—it needs to interact well with the rest of the cast (other software). Synchronous communication is like live improv: suitable for real-time, high-stakes scenes, but also demanding and prone to errors. Asynchronous communication is like rehearsed dialogues: less stress, more flexibility, but might not capture the immediacy of live interaction. Hybrid communication can offer the best of both worlds, but you'll need to balance the two to ensure a coherent performance. Tools like RabbitMQ or Celery can help orchestrate these interactions (Celery - distributed task queue, RabbitMQ - message queue).

Maintaining Machine Learning models is an ever-changing field within the industry. It covers a v riety of areas, including ML Engineering, MLOps, and platform observability. To ensure proper upkeep, we need to establish protocols for the following: Data - Keeping track of features and their lineage - Establishing data lineage for training data - Monitoring data and concept drift - Observing data pipelines Model - Tracking model metrics - Implementing version control and approval processes - Ensuring security (model theft, inference attacks, etc.) Inference Pipelines - Conducting unit tests - Testing model inference consistency - Maintaining observability Modern Data and AI tools offer the necessary capabilities.

Just to add, versioning here doesn't just mean code versioning but also dataset versioning and model versioning. Without this we do not know how models improve or degrade or if there is some data or concept or model drift.

As we deep-dive into Machine Learning, we shouldn’t undermine the importance of the interpretability and explainability of our models Having high performing models is very desirable but this often lead us to more complex algorithms, however, these models may not be really explainable. In many real-world business use cases, understanding the "why" behind a machine learning model's decision is of paramount importance. Regulatory requirements, ethical considerations, or simply the need for model transparency might demand that we can explain how and why a particular decision was made. This is where the concept of interpretability and explainability steps in to bridge the gap between complex models and human comprehension.

The nature of the ML models might add unexplainable business rules to an ontherwise well known process. Depending on the nature of a black box model, the system may behave in ways that can not be easily explained. Thus it is important to choose a model that is at least as transparent as the whole process needs to be - when handling user conversations, a black box model might be acceptable, but when making manufacturing plans or routing vehicles or evaluating financial transactions, the model used must be "able to explain it's decisions".

A machine learning model is just a component of an application or product. It’s important to remember that the product itself and the user experience is vital. For example - if the product or UX has a lot of room for improvement and no one uses it, then it doesn’t really matter if the model is amazing because it won’t get used in the first place. A model can’t always make up for a poor product. However, it can really help enhance or lower the user experience. If the model works well, then users will enjoy the product. If it doesn’t work well, complaints will be made.