Machine learning opens up new worlds for developers

*Ahmaddeen*

Survey shows continuing expansion of data scientists roles, but who's around to fill them?

By Joe McKendrick for Service Oriented |December 14, 2019 -- 16:08 GMT (08:08 PST) | Topic: Artificial Intelligence

The continuing -- but slow -- embrace of AI and machine learning means more work in designing and building models and underlying systems. These types of projects will increasingly be performed by IT departments, as the growth of data scientists is tapping out. 

That's the conclusion of a survey of 750 technology managers and professionals, released by Algorithmia, which examined growth and staffing patterns in machine learning initiatives. The survey's authors conclude companies aren't necessarily ramping up on their data science staffs, but those staff members are getting busier. 

This is opening up new opportunities for those with related skills. At least 19% report having more than 50 data scientists on staff -- up from nine percent in the survey from a year ago. Those roles are growing rapidly across all industries, the report observes. With data scientists in extremely short supply, this means pressing existing staff into these roles. "The overall lack of data science resources will result in an increasing number of developers becoming involved in creating and managing machine learning models. This blending of roles, will likely lead to another phenomenon related to this finding: more role names and job titles for the same sorts of work." 

New types of AI and machine-learning jobs being created include the following:

• Machine learning engineer
• ML developer
• ML architect
• Data engineer, machine learning operations (ML Ops)
• AI Ops

The success of machine learning initiatives depends on where one sits, the study also shows. A majority, 58%, said their efforts are successful if they produce ROI, reduce customer churn, aid in product adoption, or promote brand fidelity. Another 58% said machine learning efforts are successful when model accuracy, precision, speed, and drift meet threshold. 

These measures of relative success vary by role in the enterprise, the survey's authors report. "The individual contributor level -- data scientist, software developer -- values technical measures of ML success more so than the business metrics." At the same time, C-level executives and VPs "generally place more value on the opposite -- measuring ML success by how it ultimately benefits the company at a strategic level."

For IT and line-of-business directors, there's a bit of both business and technical metrics in play. Mid-level managers and directors value both the business unit impact -- ROI, budgetary, strategic planning metrics -- as well as the more technical metrics surrounding model performance. The Algorithmia authors predict that the manager and director level "will prove to be the crux of ML decisions made within organizations in the coming years as they seek to demonstrate their teams' capabilities but also prove to senior management that ML is a worthwhile investment to make."

'Learning' is still the operative word in machine learning initiatives

Only nine percent of machine learning efforts are mature and delivering results. At issue: scale, and models becoming obsolete too quickly.

The hype machine for AI and machine learning has been going full throttle, and one can be forgiven for thinking that every organization from the mega-techs to the corner store is turning over processes or decisions to AI. If you're still stuck trying to figure out how AI and machine learning can fit into your operations, don't worry -- so is everyone else, actually. Companies may be increasing their investments in machine learning and machine learning development, but, for the most part, are still in the early learning stages. 

Photo: 

That's the major takeaway from a survey of 750 technology managers and professionals released by Algorithmia, which specializes in such things. Survey respondents represent companies that are actively engaged in building machine learning lifecycles. While companies are increasing their machine learning investments, challenges ranging from model deployment to scaling and testing persist. (The impact on job roles explored in this related post.)

The report's authors point to challenges in moving machine learning forward, including scale, version control and model reproducibility, as well as getting senior executive buy-in. In addition, time is often not on the side of ML projects. "At companies of all sizes, data scientists spend at least 25% of their time deploying models, which indicates that a quarter of data science capability is lost to infrastructural challenges. Data science teams need to be able to deploy their work as quickly as possible to prevent their insights from being overcome by events; models and data change quickly as do market opportunities. As such, an insight that comes 10 days too late is overcome by events, and no longer useful." 

The report points to some of the various components of AI and machine learning in play, such as computer vision and new iterations of machine learning frameworks, including PyTorch 1.0, an open source machine learning framework, and TensorFlow 2.0, the open source framework originated from Google. The report's authors even mention some machine learning involving quantum computers taking place, but that's getting too far ahead of things. Hardware for AI and ML applications is also progressing, the report adds.

While the tools are falling into place, machine learning development remains in the early stages in most enterprises, the report's authors reiterate. "The model deployment lifecycle needs to continue to be more efficient and seamless for ML teams."  Only nine percent of companies report having "sophisticated models in production" for at least five years, 5+ years, up from five percent in last year's survey. Currently, another 15% indicate they are "mid-stage adopters," with models in production for between two to four years. Many of these leaders are software and IT firms, the survey shows.     

Companies with established machine learning deployment lifecycles are benefiting from measurable results, including cost reductions, fraud detection, and customer satisfaction."  Cost savings are seen as the main benefit of machine learning, which likely is based on less programming time involved in adjusting algorithms, as well as reduced payroll spent for manual tasks. Emerging in second place are the benefits the marketing and sales departments are pushing for, including generating customer insights and intelligence, as well as improving customer experience. "Undoubtedly, there are countless ways companies can apply machine learning to a particular business problem," the report's authors indicate. "For example, ML can be used to run prediction modeling to make assessments about customer churn, or to apply natural language processing to millions of tweets to analyze positive sentiment."

There are two metrics that are being applied to measure success in AI and machine learning initiatives, the study's authors find:  traditional business metrics, such as guaranteed ROI, and a more technical evaluation of machine learning model performance. A majority, 58%, said their efforts are successful if they produce ROI, reduce customer churn, aid in product adoption, or promote brand fidelity. Another 58% said machine learning efforts are successful when model accuracy, precision, speed, and drift meet threshold.