For software developers, it may be hard to comprehend, but last year the Agile Manifesto celebrated 20 years since it was conceived. Likely due to the ongoing Covid pandemic, there was not much celebration to highlight this milestone, but back in 2001 seventeen brilliant minds came together in Snowbird, Utah, to write down the first version of the manifesto on which the Agile development movement was based upon.
Most enterprise software developers and architects didn’t care much about these ideas at the time, but they were quickly embraced by creative minds in startups and academia. The early adopters looked at the way large-scale enterprises developed products and decided that their methods were not the way to create the innovative applications of the future.
Back to our present day and the Agile way of working is rippling down into the development processes for electronics and mechanics hardware. Many traditional hardware-producing companies, manufacturers of everything from cars to door locks, have started seeing the benefits of Agile development and as a result, have implemented large-scale Agile enterprise frameworks for all their engineering teams, including electronics and mechanics.
In this post, I will:
- Explain the weaknesses in the sequential “Waterfall” Model way of working that triggered the Agile way of working with software.
- Some of the basics around how Agile development works, as well as introduce large-scale enterprise frameworks.
- Clarify some common misconceptions about what Agile development means for the development organization
- Explain how the Agile methodology can be applied to electronics and mechanics development and why strategic modularization is needed to be successful.
With that said, let’s start with some concepts that involve software development and then gradually move toward how this can be applied to hardware.
Weaknesses with the Sequential “Waterfall” Model of Working
To understand why the Agile Manifesto was created, and the Agile movement was started, it is important to look at what was the most successful development method at the time. The “Waterfall” Method, often extended with the V-model, was the main development methodology for centuries and had been used to create many great products and systems up to that point. The ideas for the Waterfall Method were first mentioned in 1956 and documented in the 1970s (in an article by Winston W. Royce) and the V-model for verification of the system was invented in the 1980s. These concepts have been employed by product development for so long, it is hard to comprehend professional development without them.
Although many improvements have been made to the original “Waterfall” Model, the basics are still in place, as described in the illustration below.
Figure 1. Waterfall development model
The reason this method is referred to as the “Waterfall” Model is that all phases of the model are done in a series, ideally with no looking back.
The main value of this method is that discovering errors early in the process can be less costly than discovering the errors further along. For instance, it can be 5-7 times more costly to find errors in the coding phase than in the requirements gathering phase and 10-20 times as costly to find them in the testing phase (Source: Error cost escalation through the project life cycle - NASA Technical Reports Server (NTRS) and The Economic Impacts of Inadequate Infrastructure for Software Testing - NIST Planning Report 02-3).
As an extension to the Waterfall development process, the V-model grew out of the need to complete testing development in parallel with system design. The idea was that this would save testing time at the end of the project compared with the original waterfall model.
Figure 2. The V-model development process
As stated earlier, this model has been successful in many professional development projects in the past, but some drawbacks became evident in the decades that followed the introduction of the Waterfall and V-models:
- Testing is done only at the end of the process. While in theory, this sounds like a good idea, finding errors at this phase can severely delay and increase the cost of the project. The larger the software system, the more errors will be injected. An industrial robot controller can for instance, easily consist of 5-10 million lines of code while a modern car has more than 100 million lines of code. To attain 100% error-free for that much code the first time is simply not humanly possible. Therefore, concepts with continuous and iterative units, sub-system, and system testing have been introduced in software development to mitigate this. This was however not a part of the original V-model.
- For larger systems, it is a lengthy process to gather all the requirements at the beginning of the project. Customers also tend to not know what they want before they see it (seeing is believing). There is a high likelihood that even if you allocate 20-40% of the total project time to the requirements gathering phase at the beginning of the project, you’ll still need to spend a lot of time at the requirements change stage due to an ever-changing world and the desires of customers evolving. The longer the project is, the larger the delta between the original and the final requirements. Since the customer is most often “king”, it is very difficult to avoid these revisions without severely impacting the project schedule.
- The speed of innovation and technology can also drastically change the requirements while the product is still in development. Even if the project is successful in gathering the most important requirements at the beginning of the project, there is a high likelihood that the competitive landscape will have changed the game entirely once the product is ready.
- To mitigate the issues in changing requirements (according to reason 3 above), attempts are often made at trying to think of every conceivable future direction that the system could take and then try to design for these variables. The chief problem with this approach is that it easily creates unnecessarily complex solutions for problems that never transpire. This is one of the reasons why the word “platform” and “framework” is ill-conceived in some cases. There are simply too many platforms and frameworks out there that were created for a level of flexibility and extensibility that has never been realized in the real world.
The Agile Manifesto
The Agile Manifesto was written down in 2001 as a unifying philosophy for Agile development frameworks, such as Scrum, Extreme programming, or Feature-Driven Development.
The Agile Manifesto states the following:
“We are uncovering better ways of developing software by doing it and helping others do it.
Through this work, we have come to value:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.”
While this may appear fluffy today, it is because so much of our development has already been influenced by this philosophy. Human interaction with stakeholders is to be preferred compared to only looking at spreadsheets and digital tools. It is also rather well known that in order to create great products it is important to have an ongoing discussion with the customer instead of trying to stipulate everything into a time-consuming contract negotiation.
Since projects tend to be long in duration, it is important to have a system in place to easily embrace a potential change of requirements rather than trying to avoid them. One suggestion to avoid delays further along in the development process is to employ the use of prototypes. Being able to see and touch a product enables better decisions on the direction of its development.
In Agile development, a “leaner” requirements-gathering process is therefore assumed, since it is known that requirements will evolve during the project lifetime. Agile planning involves an incrementally increasing “backlog” of requirements that evolve. Agile also involves early prototyping, supplying a demo for stakeholders or customers to provide feedback early.
Agile Development for Enterprise
In the beginning, many startups and open-source projects started to adopt Agile thinking and methodology, and quite soon, were able to deliver successful products to the market.
This success attracted larger corporations, but since many of their products required certification and documentation to meet legal standards, such as standards pertaining to safety, electrical or environmental, they needed to find a way to work with Agile in a way that produced the documentation necessary to legally put their products on the market.
Larger corporations also employed more developers than what was possible to fit into a scrum team, which should be somewhere between 6-10 developers in size according to the famous two-pizza team size rule defined by Amazon founder Jeff Bezos.
There was, therefore, a need to create an Agile development model that could scale to many more developers. Thus, Enterprise Agile development methods were born, for example, the Scaled Agile Framework (SAFe), Large-Scale Scrum (LeSS), or Disciplined Agile Delivery (DAD).
These development frameworks embrace the Agile development philosophies, but in a way that enables them to scale up to match the large-scale projects that some companies require. For instance, if your company is developing a car.
In the Scaled Agile Framework, an Agile Release Train (ART) is defined, to which several scrum teams deliver code.
Figure 3. Scaled Agile Framework Agile Release Train
The ART has a prioritized backlog of tasks, from which an increment backlog is pulled (from the top). The increment backlog contains the most prioritized tasks for the ART and is typically 1.5 times larger than what the teams can meet. Each scrum team will then pull tasks from the increment backlog to their team backlog and plan the workload accordingly for the coming increment covering 10 weeks (4 work iterations + 1 reflection and improvement iteration).
For more complex products, there might be a need to have more than one ART to deliver a product, therefore there the use of a Solution Train is employed. The Solution Trains can host many ARTs simultaneously. For example, some automotive manufacturers have more than 40 ARTs in one Solution Train.
Figure 4. Scaled Agile Framework Solution Train
There can be several Solutions Trains within a company, and they are in turn controlled by Lean budgets, which define the resources that the Solution Trains consume.
These frameworks were created for software development, but nothing is preventing other types of development from also using them, for instance, electronics or mechanics development. Many of the roles that are defined for software development exist also in electronics and mechanics development or could easily be customized for these organizations.
Some Common Misconceptions about Agile Development
It’s not uncommon for those that haven’t experienced working in Agile development teams to have some misconceptions about what it means. Below I will try to clarify some of the misconceptions I have seen the most.
Agile development is unstructured
Since not all product requirements are available up-front with Agile development, some tend to believe that agile is just a means to do “happy-hacking” speculatively, without any specifications. This is very untrue. Depending on which Agile development method you use, you’ll find that most of them are very structured.
To start, there needs to be a prioritized backlog of items to develop. This prioritized backlog should be the basis for the iteration planning with the intention to only pick the requirements that are on top of the stack. In the Agile way of working, this prioritized backlog should be constantly refilled and reprioritized depending on new input from the market, engineering, or competitors.
In Scrum, each person has a role (typically; Scrum-master, Product owner, or team member) and there is a specified outline for conducting a daily-scrum meeting (standing up for instance). Iterations have a set time limit, typically 2-3 weeks and no code should be checked in without compliance with a defined “Definition of Done” (DoD). These checkboxes may include static code analysis, test-building all possible builds, buddy checking, unit-testing, etc. For hardware development, the DoD will of course need to be modified to comply with best practices for that development environment.
In Agile development, you still must create a plan for your resources and the tasks at hand. This process can involve “Kanban” boards, as well as work burn-down charts to track the progress. The work burn-down statistics are later used to calculate the team velocity for planning with more preciseness in the future.
Agile means less administration & documentation and more coding
Many that are new to the Agile development methods are surprised to find that in Agile, there is a substantial amount of administration and documentation that occurs as part of its framework.
Before coding begins there are still the requirements of documenting your intention with the new design and defining the DoD. The main difference to traditional development methods is that the documentation is limited to the current iteration scope without the need to take the full scope of the product into account. The result of the development should be very similar, with the exception that Agile development methods only cover the completed iterations. After all, there is no need to document areas that will not be developed, thereby eliminating “waste” documentation.
With the mandate to hold daily standup planning meetings as part of many Agile methods, the corresponding result can be a significant amount of administration at the end of a project. The main difference is that in Agile, the scope of the administration is limited to the current iteration or increment goals. In this way it is possible to quickly react to modifications within the team, shift resources to areas that were more complex than expected or move goals out that cannot be completed as expected.
The Scrum master and product owner should also continuously work with the prioritized backlog so that the most important tasks are always on top when the team has completed the current iteration or increment.
If there many Agile development teams are working with the same products, for instance in an agile release train, the teams will also need synchronization of task progress. In this way, parallel teams work optimally and avoid costly delays due to the poor planning of tasks. This alignment typically occurs in the weekly “Scrum-of-scrums”, where tasks that rely on other teams are coordinated. This same meeting is the best venue to also request help from other teams in completing tasks that are more difficult than what was initially forecasted from the team planning stage.
Figure 5. Continuous meetings in the agile development methods
Agile development is less transparent than the traditional way of developing
Because tasks are broken out into iterations or program increments, typically documented on a Kanban board, and the team capacity is known, it is obvious when a team has reached its max workload capacity or when there is space to take on additional work. Whereas in traditional Waterfall development methods, the time estimations usually don’t consider the velocity of the teams or in what order the requirements are implemented. In Agile, a product owner or manager can glance at planning boards and in an instant, know the status of a team’s current tasks and what are their next priorities.
Figure 6. Continuous meetings in the agile development methods
Agile Development for Hardware Platforms
It can be noted that only one of the sentences in the Agile Manifesto mentions software explicitly, “working software over comprehensive documentation”. This provides an insight that implementing Agile for hardware might not be such a stretch after all.
The process for developing electronics and mechanics has a “waterfall mentality” in the sense that they are too linear and getting all the requirements right, as well as more testing upfront, would benefit this type of development too.
Creating prototypes of hardware is a well-known concept. However, there can be very long lead times to create a full printed circuit board (PCB) with soldered components (PCBA), or for that sake, die-casted or machined mechanic structures. Due to advancements in CAD, simulation, and CPU/GPU processing performance, digital prototypes can be made in a reasonable amount of time today. Additive manufacturing (i.e. 3d-printing) and collaborative robotics are also making it possible to create real prototypes of mechanical and building structures very fast without spending time on shipping.
This leads to the possibility that hardware prototypes can be built frequently, at least daily or weekly, thereby shortening the feedback loop on the mechanical and electrical design for these complex systems. The previously mentioned automotive manufacturer is building a complete car digitally every day using these technologies to accelerate the speed of development and agility.
Minimum Viable Product and Minimum Marketable Product for hardware development
The idea of creating requirement backlogs for mechanics and electronics is also not far-fetched. In Agile development, it is also important to identify a Minimum Viable Product (MVP); the bare minimum development of a product that is feasible to share with a customer for feedback. This means that in the backlog you put the features that are essential to reach the MVP on top and add the more advanced features/parts further down the backlog.
Once the customer and stakeholder feedback on the MVP has been received, you can improve the backlog to define a Minimum Marketable Product (MMP). An MMP is the minimum set of requirements needed before a product can be put on the market and start earning money, i.e., the fastest path toward profit.
While this is great in theory, there is a fundamental difference between the development of electronics and mechanics to software. That is the ability to efficiently improve the product over time with feature updates, even after the product has left the factory. This is a feature that is utilized in software to a very large extent and often automatically through an Internet connection, i.e., over-the-air (OTA) software updates.
Since these updates cannot be done for electronics and mechanics, the scope for the MVP would be larger than what it would be for a pure software product. It however still makes sense if you consider the product configurations and a modular way of working.
Product configurations are utilized heavily in modular thinking where a number of “base” module variants are developed first, and additional variants that are necessary for other product configurations are developed later.
Let’s look at this fictive example with an electrical winch.
Figure 7. Electrical winch
The winch is made from a modular product platform, with an architecture consisting of abstract modules that have several physical module variants:
Figure 8. A modular architecture describing the winch
The modules in the winch can be combined into several sales configurations to reach different market needs. The module variants that are necessary to create the first sales configuration would be the definition of the MVP for the module system, i.e., the module variants that need to be developed first. All the other module variants can be developed later, and reuse some of the other modules from the MVP.
Figure 9. The MVP for a module system is the 1st configuration
To succeed with Agile hardware development your architecture needs to be modular
Based on the MVP and MMP discussion for electronics and mechanics, it becomes clear that to be successful with Agile development practices a product must be developed in a modular way. This applies to any product, whether it is based on mechanics, electronics, software, or any combination of these technologies.
Designing stable interfaces between modules creates a separation of concern that allows the module variants to be developed in parallel by different Agile teams. The MVP module variants for the 1st release need to be developed first since they are all needed to create a product. These can however be developed in the order that enables the most efficient development for Agile and self-organizing teams.
Agile development is here to stay, and recent data from Gartner (sources: Future of Work Trends: Teams Become Agile and Hype Cycle for Strategic Portfolio Management, 2021) shows that:
- Interdisciplinary teams can iterate, innovate, and deliver fast.
- In a Board of Directors Survey 2021, 69% responded that they wanted to accelerate digital business initiatives.
- The traditional waterfall approach, combined with a hierarchical “command and control” decision-making process, is too slow and inflexible to meet this requirement.
- By 2026, 75% or more of organizations will have interdisciplinary teams following Agile principles
- up from at least 25% in 2020
The strongest modular product architecture is connected with the overall business strategy. Combining this business strategy with Agile development principles has the potential to accelerate the company into a prosperous and well-oiled machine with unprecedented speed to market with products that are attractive to your customer's needs now and in the future.
Want to Know More?
If you’d like to learn more about implementing Agile principles into your hardware development, check out our webinar on Best Practices for Software Architecture in Hardware Companies. Additionally, feel free to message me using the contact information below or use our contact form.
AUTHOR
Roger Kulläng
Senior Specialist Software Architecture
+46 70 279 85 92
roger.kullang@modularmanagement.com
