Best Practices in Machine Learning Infrastructure

aib19-1

Developing processes for integrating machine learning within an organization’s existing computational infrastructure remains a challenge for which robust industry standards do not yet exist. But companies are increasingly realizing that the development of an infrastructure that supports the seamless training, testing, and deployment of models at enterprise scale is as important to long-term viability as the models themselves.

Small companies, however, struggle to compete against large organizations that have the resources to pour into the large, modular teams and processes of internal tool development that are often necessary to produce robust machine learning pipelines.

Luckily, there are some universal best practices for achieving successful machine learning model rollout for a company of any size and means.

The Typical Software Development Workflow

Although DevOps is a relatively new subfield of software development, accepted procedures have already begun to arise. A typical software development workflow usually looks something like this:

aib19-2

This is relatively straightforward and works quite well as a standard benchmark for the software development process. However, the multidisciplinary nature of machine learning introduces a unique set of challenges that traditional software development procedures weren’t designed to address.

Machine Learning Infrastructure Development

If you were to visualize the process of creating a machine learning model from conception to production, it might have multiple tracks and look something like these:

aib19-3

Data Ingestion

It all starts with data.

Even more important to a machine learning workflow’s success than the model itself is the quality of the data it ingests. For this reason, organizations that understand the importance of high-quality data put an incredible amount of effort into architecting their data platforms. First and foremost, they invest in scalable storage solutions, be they on the cloud or in local databases. Popular options include Azure Blob, Amazon S3, DynamoDB, Cassandra, and Hadoop.

Often finding data that conforms well to a given machine learning problem can be difficult. Sometimes datasets exist, but are not commercially licensed. In this case, companies will need to establish their own data curation pipelines whether by soliciting data through customer outreach or through a third-party service.

Once data has been cleaned, visualized, and selected for training, it needs to be transformed into a numerical representation so that it can be used as input for a model. This process is called vectorization. The selection process for determining what’s important in the dataset for training is called featurization. While featurization is more of an art then a science, many machine learning tasks possess associated featurization methods that are commonly used in practice.

Since common featurizations exist and generating these features for a given dataset takes time, it behooves organizations to implement their own feature stores as part of their machine learning pipelines. Simply put, a feature store is just a common library of featurizations that can be applied to data of a given type.

Having this library accessible across teams allows practitioners to set up their models in standardized ways, thus aiding reproducibility and sharing between groups.

Model Selection

Current guides to machine learning tend to focus on standard algorithms and model types and how they can best be applied to solve a given business problem.

Selecting the type of model to use when confronted with a business problem can often be a laborious task. Practitioners tend to make a choice informed by the existing literature and their first-hand experience about which models they’d like to try first.

There are some general rules of thumb that help guide this process. For example, Convolutional Neural Networks tend to perform quite well on image recognition and text classification, LSTMs and GRUs are among the go-to choices for sequence prediction and language modeling, and encoder-decoder architectures excel on translation tasks.

After a model has been selected, the practitioner must then decide which tool to implement the chosen model. The interoperability of different frameworks has improved greatly in recent years due to the introduction of universal model file formats such as the Open Neural Network eXchange (ONNX), which allow for the porting of models trained in one library to be exported for use in another.

What’s more, the advent of machine learning compilers such as Intel’s nGraph, Facebook’s Glow, or the University of Washington’s TVM promise the holy grail of being able to specify your model in a universal language of sorts and have it be compiled to seamlessly target a vast array of different platforms and hardware architectures.

Model Training

Model training constitutes one of the most time consuming and labor-intensive stages in any machine learning workflow. What’s more, the hardware and infrastructure used to train models depends greatly on the number of parameters in the model, the size of the dataset, the optimization method used, and other considerations.

In order to automate the quest for optimal hyperparameter settings, machine learning engineers often perform what’s called a grid search or hyperparameter search. This involves a sweep across parameter space that seeks to maximize some score function, often cross-validation accuracy.

Even more advanced methods exist that focus on using Bayesian optimization or reinforcement learning to tune hyperparameters. What’s more, the field has recently seen a surge in tools focusing on automated machine learning methods, which act as black boxes used to select a semi-optimal model and hyperparameter configuration.

After a model is trained, it should be evaluated based on performance metrics including cross-validation accuracy, precision, recall, F1 score, and AUC. This information is used to inform either further training of the same model or the next iterate in the model selection process. Like all other metrics, these should be logged in a database for future use.

Visualization

Model visualization can be integrated at any point in the machine learning pipeline, but proves especially valuable at the training and testing stages. As discussed, appropriate metrics should be visualized after each stage in the training process to ensure that the training procedure is tending towards convergence.

Many machine learning libraries are packaged with tools that allow users to debug and investigate each step in the training process. For example, TensorFlow comes bundled with TensorBoard, a utility that allows users to apply metrics to their model, view these quantities as a function of time as the model trains, and even view each node in a neural network’s computational graph.

Model Testing

Once a model has been trained, but before deployment, it should be thoroughly tested. This is often done as part of a CI/CD pipeline. Each model should be subjected to both qualitative and quantitative unit tests. Many training datasets have corresponding test sets which consist of hand-labeled examples against which the model’s performance can be measured. If a test set does not yet exist for a given dataset, it can often be beneficial for a team to curate one.

The model should also be applied to out-of-domain examples coming from a distribution outside of that on which the model was trained. Often, a qualitative check as to the model’s performance, obtained by cross-referencing a model’s predictions with what one would intuitively expect, can serve as a guide as to whether the model is working as hoped.

For example, if you trained a model for text classification, you might give it the sentence “the cat walked jauntily down the street, flaunting its shiny coat” and ensure that it categorizes this as “animals” or “sass.”

Deployment

After a model has been trained and tested, it needs to be deployed in production. Current practices often push for the deploying of models as microservices, or compartmentalized packages of code that can be queried through and interact via API calls.

Successful deployment often requires building utilities and software that data scientists can use to package their code and rapidly iterate on models in an organized and robust way such that the backend and data engineers can efficiently translate the results into properly architected models that are deployed at scale.

For traditional businesses, without sufficient in-house technological expertise, this can prove a herculean task. Even for large organizations with resources available, creating a scalable deployment solution is a dangerous, expensive commitment. Building an in-house solution like Uber’s Michelangelo just doesn’t make sense for any but a handful of companies with unique, cutting-edge ML needs that are fundamental to their business.

Fortunately, commercial tools exist to offload this burden, providing the benefits of an in-house platform without signing the organization up for a life sentence of proprietary software development and maintenance.

Algorithmia’s AI Layer allows users to deploy models from any framework, language, or platform and connect to most all data sources. We scale model inference on multi-cloud infrastructures with high efficiency and enable users to continuously manage the machine learning life cycle with tools to iterate, audit, secure, and govern.

No matter where you are in the machine learning life cycle, understanding each stage at the start and what tools and practices will likely yield successful results will prime your ML program for sophistication. Challenges exist at each stage, and your team should also be primed to face them.

Source: https://blog.algorithmia.com/best-practices-in-machine-learning-infrastructure/

Financial Services and the cloud: Accelerating the compliance journey

In the financial services industry, there is a growing interest in the cloud and its advanced capabilities to improve existing operations and innovate and transform business. Yet, when it comes to cloud adoption and implementation, there is still a lot of confusion. The most common misperception: Regulation is a barrier to the cloud.

Debunking the cloud myth

Our team at Microsoft has spent the last seven years working closely with financial services regulators and found the opposite to be true. Regulations are tech neutral to cloud computing, and our experience is that regulators are more open to cloud technology than when we started this journey years ago. That said, there is a lot of banking regulation that comes into play when using cloud services, and both banks and regulators want to get it right. Regulators are also modernizing their laws to address cloud computing.

The role of technology vendors

As banks look to third-party vendors for cloud services, our role goes beyond providing a scalable platform they can use to run and operate their business. Technology providers are also responsible for helping them understand the cloud journey; providing both financial services organizations and regulators with transparency in how they manage and operate their cloud services; and ensuring their customers have the control and security of their data to meet their compliance obligations.

Accelerating the compliance journey

As a global cloud service provider, Microsoft has made significant investments in helping the financial services industry meet and manage its regulatory responsibilities and accelerate the compliance journey.

Here are a few ways we are doing this, read full blog here:

Discover your journey in Cloud Computing career, by preparing on following courses:

Where Cloud Computing Jobs Will Be In 2019

Demand for cloud computing expertise continues to increase exponentially and will accelerate in 2019. To better understand the current and future direction of cloud computing hiring trends, I utilized Gartner TalentNeuron. Gartner TalentNeuron is an online talent market intelligence portal with real-time labor market insights, including custom role analytics and executive-ready dashboards and presentations. Gartner TalentNeuron also supports a range of strategic initiatives covering talent, location, and competitive intelligence.

Gartner TalentNeuron maintains a database of more than one billion unique job listings and is collecting hiring trend data from more than 150 countries across six continents, resulting in 143GB of raw data being acquired daily. In response to many Forbes readers’ requests for recommendations on where to find a job in cloud computing, I contacted Gartner to gain access to TalentNeuron.

Find how many Cloud Computing jobs are standing in the current market, read here:

Find your career in Cloud Computing by joining right course:

Digital Transformation and Future Workforce

Digital transformation is upon us, and every industry and every business is part of it. Change is happening fast and many, if not all, industries are being redefined. Manufacturing, especially, has huge gains to realize from this disruption, as advanced technologies like IoT, artificial intelligence (AI), machine learning, mixed reality, digital twins, and blockchain empower manufacturers to improve efficiency, flexibility, and productivity through new levels of intelligence.

Of all the technologies reshaping our industry, I would say that AI plays one of the biggest roles here in terms of the disruption—and resulting opportunity—for both our industry and our workforce. AI has made large strides in recent years and is poised to open up new types of employment opportunities. The transformational possibilities of AI for manufacturers, employees and the industry-at-large are enormous.

However, as we look to a future powered by a partnership between computers and humans, it is important that we work to democratize AI for every person and every organization – something Microsoft is deeply focused on.

Two critical data points, from recent research, to establish the enormity of change:

  • Machines complete 29% of tasks today, which is expected to go up to 71% by 2025.
  • 75 million jobs to be displaced by automation and 133 million new jobs to be created between 2018-2022.

Read full blog here:

You can join following course to improve your knowledge of how digital transformation effects the future world:

Azure Global Infrastructure

Achieve global reach and the local presence you need

Go beyond the limits of your on-premises datacenter using the scalable, trusted and reliable Microsoft Cloud. Transform your business and reduce costs with an energy-efficient infrastructure spanning more than 100 highly secure facilities worldwide, linked by one of the largest networks on earth.

Deliver services confidently with a cloud you can trust

 

 Scale globally- Reach more locations, faster, with the performance and reliability of a vast global infrastructure.
 Safeguard data- Rely on industry-leading data security in the region and across our network.
 Promote sustainability- Help build a clean-energy future and accelerate progress toward your sustainability goals.

Let Azure keep your data secure

Azure safeguards data in facilities that are protected by industry-leading physical security systems and are compliant with a comprehensive portfolio of standards and regulations.

Read the full blog here: https://azure.microsoft.com/en-in/global-infrastructure/

Join a sustainable IT future by joining the following courses:

Azure Infrastructure Specialist

Azure Developer

Technical Architect

Access training for New Azure certifications

AZ-301 Microsoft Azure Architect Design Training

Installation, Storage, and computer with Windows Server 2016 – 70-740

Networking with Windows Server 2016 – 70-741

Identity with Windows Server 2016 – 70-742

 

What is cloud computing?

Simply put, cloud computing is the delivery of computing services—servers, storage, databases, networking, software, analytics, intelligence and more—over the Internet (“the cloud”) to offer faster innovation, flexible resources, and economies of scale. You typically pay only for cloud services you use, helping lower your operating costs, run your infrastructure more efficiently, and scale as your business needs change.

Top benefits of cloud computing

  • Cost
  • Speed
  • Global scale
  • Productivity
  • Performance
  • Security

Types of cloud computing

  • Public cloud
  • Private cloud
  • Hybrid cloud

Types of cloud services:

  • IaaS
  • PaaS
  • serverless
  • SaaS

Also know more about, How cloud computing works & Uses of cloud computing,

By clicking on following links:

https://azure.microsoft.com/en-us/overview/what-is-cloud-computing/https://www.infoworld.com/article/2683784/what-is-cloud-computing.htmlhttps://azure.microsoft.com/en-in/overview/future-of-cloud/

You can improve your cloud computing knowledge by joining following course:

Uncovering the ROI in AI

Artificial intelligence is loosely defined as a broad set of technologies that think and act like people. These intelligent machines can understand, interpret, reason and engage people in a natural way, as a human would do. It’s clear from working with clients across several industries that there are different AI maturity levels. AI maturity isn’t just about the technology you use; it’s also determined by the people you have in place and the supporting business processes. Understanding what you’re doing today gives you a clear starting point, so you know where to focus your efforts.

Finding the ROI in AI

ai blog 8

In the market, there is variety in the number and types of problems that AI can solve, and it will be rare that an organization will buy or even build one AI product that will be able to solve all its problems. Instead, companies will be able to better maximize their ROI by applying different AI technologies to their existing specific business problems. Today, we see several companies finding the biggest ROI with the combination of automation and cognitive service technologies.

For example, Avanade has been working with an insurance company that began by automating rote, manual processes to prioritize claims and assign them to workflows. The company took the next step by adding cognitive services that could read all customer query tickets and understand the customer’s intent. The queries were then categorized and assigned to the right workflow. By using AI to answer more standard tickets, the company liberated people to answer more complex questions, resulting in up to 60 percent gains in efficiency and resolution time.

This example shows that you don’t need to take big steps to see positive results from AI implementations. Simply find the right process, get started and then evolve your approach to deliver business results.

Solve the problem first, then apply the right technology

While it’s easy to view AI as a single technology, it is in fact various technologies that range from automating with cognitive services, all the way to advanced analytics and deep learning to proactively solve problems.

Today, many organizations begin their AI implementation with automation, in areas with lots of manual and repeatable tasks like call centers or back office process like finance and accounting. It’s important to understand the human processes and behaviors that are driving your business and decide how AI can augment them, not replace them. Think less in terms of the technology, and more about the impact that you want AI to have on the people connected to your business – both customers and employees. This is human-centered AI – an approach that focuses on augmenting the workforce to improve customer and employee experiences.

For example, a financial services company in Europe was finding that they were losing a significant number of customers to their competitors. They engaged with Avanade to create machine learning models to better predict customer churn. Looking over a 3-month period and over 100 factors as inputs, the model was able to predict which customers were most likely to churn, and the financial services firm was able to take appropriate action and targeted messages in their next marketing campaigns, reducing the numbers of customers likely to churn by 50 percent.

Shifting to AI-first

Across almost every industry, we are seeing significant ROI for those organizations applying strategic AI technologies. Some of that is quantitative, while others are more qualitative, such as better customer and employee experiences, tracked through net promoter scores and other similar engagement measures. And while benefits are being realized, the true leading companies are beginning to determine how to apply AI more holistically outside of the siloed projects running within specific business units throughout their company.

These leaders are recognizing how to bake AI into their business, across all core functions. Stitch Fix, a leading online and personal shopping subscription service, is a great example of a model-driven company that sold almost $1B of clothing in 2017. You can see just how pervasive the models are in every element of their business. But it’s not all automation, as it’s truly a great example of human-centered AI. Their stylists were able to utilize the data and alter or override the styling the algorithm delivered. This is important because neither the person nor the algorithm can be perfect.

Every business looking to embrace AI should be clear about what data is being used and for what purposes. According to Avanade and Wakefield’s research, 89 percent of IT decision-makers say they have encountered an ethical dilemma at work caused by the increased use of smart technologies and digital automation, with 87 percent admitting they are not fully prepared to address the ethical concerns that exist in this new era. We encourage any organization working to implement AI to create a digital ethics framework that sets out how you will manage the bias that can be inherent in any AI algorithm. This includes internally built applications and purchased solutions. To help address this, Avanade recently created an ethics task force that is developing a digital ethics framework to help us internally and to guide our clients.

Leading companies that take a holistic, AI-first approach, driven by strategic business needs, will see significant ROI in their bottom line and for their shareholders.

Source: https://www.avanade.com/en/blogs/avanade-insights/artificial-intelligence/uncovering-the-roi-in-ai