On one side, machine learning companies are pushing frameworks and other early tools as way that machine learning (ML) can change and update software solutions. On the other side, there are companies focused on software security, code coverage and other issue related to mainline software development. At this point, there is not much of an overlap, but there is a need for each side to help the other.
This article accepts a simple, obvious, and often forgotten premise: software continues to become more complex. Back in the mainframe days, manual code review was possible. That was a combination both simpler code being developed and slower change cycles. Advanced in hardware, software and user experience (UX) have massively increased both the volume and complexity of code. The web, smartphones, and other technology have both meant far more people using technology and causes a service oriented change to work to provide software updates faster.
Two acronyms and an abbreviation have taken a lot of mindshare within development circles in the last few years:
- CI: Continuous Integration, the ability to quickly and constantly add code to an existing code base
- CD: Continuous Deployment, the ability to quickly move new code into production
- DevOps: Development and Operations working more closely together, just another word for Agile, which is just another word for common sense
What those mean is that more code is coming faster while IT and ISVs want to provide code updates to their customers on a much more frequent basis.
To succeed at that, ML and developer tools must leverage each other’s strengths.
Machine Learning Leveraging Development Tools
Frameworks are only a mid-point in providing better access to ML, there are a lot more parts of advanced development environments ML must still leverage. Code coverage tools, for example, provide statistical analysis of how much of your code is used during testing and how often each segment is tested. That’s necessary to ensure key areas of functionality are well tested and that the team adequately tests the entire application.
Another critical area is security planning and testing. The concept of security covers many things, from network, to encrypting databases, providing user access levels and more. At the code level, security means thinking about security from the initial development of use cases through full testing. ML groups need to learn from the rest of the industry how to address security policies and challenges throughout the software development lifecycle (SDLC).
ML teams have primarily been in the academic arena and in sandboxes within large corporations. As companies of all sized begin to leverage the potential of ML development teams need to change from research mindset to more formal development. Potential ML customers are already using strong SDLC tools in other areas of their software infrastructure, so they will expect the same level of development from the ML firms.
Development Tools Leveraging Machine Learning
The increasing volumes of code and the faster version cycles mean that the tool companies also need help. Especially in the arena of security, there is so much to cover and test in such a short time, those firms are beginning to look at ML as a technology that can more rapidly process code while increasing accuracy of test results. Faster cycles with lower risk is a combination that can’t be ignored.
The move to the cloud is a key reason why the development tools market must adapt. The accessibility and scalability of cloud applications is what is helping push CI/CD and DevOps. The cloud is helping more people understand that the needs of the market and the advances in technology are both advancing at a pace that requires faster response. That means:
- operations being involved earlier and more consistently in the design and development process.
- developers becoming more involved in understanding actual product usage.
- that upgrade cycles must happen more rapidly.
To support the needs of the CI/CD movement, the pure performance of scale-out, cloud, computing and the increased analytics of ML can combine to support the rapid testing and analysis that will be required. Machine learning be used to better analyze more data in larger tests in a timeframe that still supports modern, rapid, development and deployment models.
The SDLC is changing. It means more code, much faster, and far more people involved earlier in the lifecycle. Managing coding projects is no longer the simple waterfall approach of the 1970s. Machine learning will play a large part in managing the SDLC in the coming decade.
Machine learning Maturity Means Both
Programming is a craft. It is part engineering and part art. The flexibility of the craft combines with the complexity of modern systems to mean that a maturing industry must learn how to better manage development. Machine learning developers need to link into the existing processes that can help them create systems that will be trusted to perform at needed levels. The development ecosystem can also turn to machine learning to help manage the advance challenges of modern applications.