Lazy loading (or 'loading on demand' or 'deferred instantiation' if you prefer) is a popular performance optimization technique. It distributes work that would be done 'upfront' over a longer duration. In some ways it is similar to the way agile approaches distribute analysis and design work throughout the duration of a project.
Lazy Loading Examples
Imagine a large tree control in a GUI panel or web-page. Loading the data for all the nodes in the tree from a database or across a network from a web server could take long enough to cause an unacceptable delay in displaying the panel or page.
Instead, we initially load only the top level nodes in the tree control. Only when a user chooses to expand one of the top level nodes to view its sub-nodes do we load the data for those sub-nodes. Loading only the top level nodes at first enables our GUI panel or web page to be displayed more quickly.
Similarly, consider a tabbed panel in a GUI or web-page that has several complex tabs. Instead of constructing all of the contents of each of tabs before displaying the tabbed control, lazy loading constructs only the first tab and only constructs the contents of the other tabs if and when the user selects that particular tab.
There are numerous different applications of the pattern from which I could pick. The two examples above are user interface design-related but they did not have to be. They could have been about restoring objects of any kind from a database or file, etc.
It is important to note that, if our user only actually expands a small proportion of the nodes in the tree-control, or only visits one or two of the tabs in the tabbed control, the total amount of data transferred from the database or across the network is greatly reduced. However, because several trips are needed to the database or to the sever, the total time taken to load the data may not be significantly smaller and could, in fact, be longer.
If our user visits every tab in the tab control and expands every node in the tree control, we end up transferring exactly the same amount of data that we would transfer if loading it all upfront. In this case, the total time taken to load the data is guaranteed to be longer because of the large number of trips to the database or server.
Despite this, our user is happier because the time taken to load the data is spread out. They do not experience a long initial, highly noticeable, delay in using the software. Instead there are several much less significant delays when expanding nodes and switching tabs that the user hopefully finds much more acceptable.
Similarities to Agile Approaches
Now compare this to the difference between traditional waterfall-like software development process and more agile, highly iterative approaches. Traditional processes do all the analysis and design work upfront. Agile approaches spread the time required to do that analysis and design work across numerous iterations.
Just like our GUI user benefitting from lazily-loaded user interface controls, the customer of an agile team does not experience a long initial delay before seeing anything delivered. Instead, they experience the frequent delivery of tangible, working results.
In the face of naive objections, it is important to stress that an agile approach does not remove the need for analysis and design. It simply distributes that work throughout the project's duration instead of lumping it all at the beginning. Anyone using 'agile' to excuse them from analysis and design work is abusing the term.
In fact, an agile project requires just as much, if not more, analysis and design as a waterfall-like project. Remember that our lazy-loading examples only load a smaller amount of data than their upfront counterparts if the user visits only a subset of the nodes or tabs. Similarly an agile approach can only do less analysis and design than a waterfall approach if, as the project proceeds, it turns out that a significant number of the original requirements of the project are not really required or are not of enough value to the customer to justify developing them. Where agile approaches do save time in analysis and design time is in the reduced need to produce and review volumes and volumes of carefully formatted documentation. Again, however, this can be taken too far and agile used as an excuse not to produce adequate documentation of analysis and design. Such behavior is not agile but simply immature and undisciplined.
There are other similarities between our lazy-loading examples and agile approaches:
- Firstly, if we load all the data from our database or across the
network to build all our user interface control contents upfront, we
can disconnect from the database or the server. They are no longer
required. The user can navigate through our controls and display all
the data possible without return trips to the database or server. In
contrast, our lazy-loaded user interface controls need to remain
connected to the database or server to be able to retrieve the data
they need 'on demand'. Similarly, the customer or customer
representatives (domain experts, business analysts, etc) specifying the
requirements for a system can be dismissed after the analysis phase in
a waterfall process but, in an agile approach, a key success criteria
is the continued presence of the customer or customer reps throughout
the duration of the project. Without the availability of the relevant
repository of information, the on-demand analysis and design of an
agile approach is doomed to failure.
- Secondly, despite the on-demand nature of the user interface
there is some initial work required. The high-level nodes of the tree
control must be laid out appropriately, the labels for the tabs in the
tabbed-control must be displayed, etc. Similarly, successful agile
projects perform just enough high-level analysis and design (using
techniques like Eric Evans Domain-Driven Design or Peter Coad's
Modeling in Color) before starting development iterations.
- Thirdly, if we have the processor bandwidth, we can speculatively load data into our user interface controls in the background while the user interacts with the nodes or tab currently being displayed. Similarly, if we have enough development bandwidth, we can speculatively perform analysis and design of requirements of future iterations while development on current iterations is in progress. Possibly, we believe that something coming up in the near future is going to need some significant research and one of the team can be spared at the moment to get a head start on that. In both cases, there is a risk that the work done may end up being not useful; our GUI user may not expand a node in the tree control that we have loaded speculatively and our customer representative may decide the features requiring the research are no longer of high enough priority to make them worth developing.
Differences to Agile Approaches
There are, of course, differences as well as similarities between lazy-loading of user interface controls and the application of agile approaches to software development. For example, it is unusual that the loading of one set of sub-nodes in the tree control would affect the contents of another node. In contrast, it is almost inevitable that analysis and design work done later in an agile project will at some point have some negative impact on analysis and design work done earlier leading to some rework.
Similarly, it is unusual if the loading of one set of sub-nodes in the tree control significantly contributes to the loading of the contents of another node in the tree. In contrast, the knowledge gained from analysis, design, implementation and testing in earlier iterations can significantly improve the analysis and design work of subsequent iterations. Agile approaches almost assume that the cumulative knowledge gained through delivering frequent, tangible, working results will significantly outweigh the amount of rework caused by subsequent iterations negatively impacting the analysis and design of previous iterations.
While the above comparison with lazy-loading reinforces some of the advantages, in general, of agile approaches over their waterfall-based counterparts, there are many additional reasons why approaches like Scrum and Feature-Driven Development are generally improvements on traditional waterfall approaches. Many of these are as much psychological as they are organizational because in the end, serious software development is so very much more about people working well together than it is about particular processes and technologies.