Construction
Software Construction involves designing, coding, testing, debugging, integration, roll out and maintenance.
Metaphors to describe software development - writing code, growing the system, building software.
An appropriate metaphor captures the observable details of the profession and is well understood by all. Science has historically moved from worse off metaphors to better ones e.g. from all planets and sun rotating around the earth, to all planets rotating around the sun. What seems appropriate today will become obsolete tomorrow.
Todo - read The Structure of Scientific Revolutions by Thomas S Kuhn
Preparations
Building Software requires an appropriate level of planning - not over or under planned. More experience with a category of projects allows us to make assumptions, foresee risks, put in place mitigation strategies, etc i.e. rolling wave planning.
Testing can't detect if the wrong product is being built or if the process of building is wrong. Testing detects flaws when software doesn't conform to the requirements. This makes initial preparations crucial.
Prepare well in advance, especially at the start of the project so that risk identification and reduction can be started up front. Preparing in advance, allows focusing on requirements, building plans to design, develop and test. The level of preparation varies along the complexity of the project.
If requirements aren't known or understood, then there are two projects. The first project is to determine requirements, refine and document them. The second project is to proceed with the design, development and further phases.
If requirements aren't known or understood, then there are two projects. The first project is to determine requirements, refine and document them. The second project is to proceed with the design, development and further phases.
Comparing to Agile/Scrum methods, preparations are building out the backlog in sufficient detail. Scrums/Iterations will allow refining the requirements or documenting new requirements, added to the backlog for further iterations. At some point the software build is considered to have sufficient features (Gmail 2004) that it can now be released to users for more inputs, test the waters.
A well prepared project -
- Defines the Problem without listing the solution
- Lists the Requirements explicitly i.e. scope, to avoid confusion or arguments at a later stage
Requirements
Requirements will change as the project progresses. The cost of changing them and the rationale behind the change should be discussed with everyone. Plan for changes, put in place a development approach that accommodates them e.g. iteratively refine a prototype into a final product. Manage the changes, don't let them ruin the project and don't guard the project so that you end up building a useless product.
Know when to dump the project - has competition made an unexpected move? has there been a technology change? economic/financial surprise? force majeure?
Checklist of questions for gathering requirements
- Functional - inputs, outputs, interfaces, user tasks
- Non-Functional - response time, throughput, memory and storage limits, maintainability
- Quality - document requirements in users language, conflict amongst requirements, level of detail, likelihood of change in a requirement
- Completeness - areas of incomplete requirements, level of comfort with the requirements
System Architecture
The step between requirements and construction. It can specify both system wide design constraints or those that apply to subsystem. The level of detail may wary. The intent is that Architecture Document provides the overall guidance to developers, allows them to partition the work, work independently and also coherently.
Broadly it covers these -
- various alternatives considered and the reason for selection one program organization over others.
- the building blocks of the system, their interfaces and communication rules.
- file formats, database, performance tweaks to access data containers.
- user experience (XD), inputs and outputs, security, managing resources, performance and scalability, internationalization and localization.
- defines how to handle errors, detection and propagation, input validation and assumptions.
- reliability of the system, techniques to backup to last good state, auxillary processes, hot vs cold standby's.
- over-engineer for superb performance versus simple solution to get it to work
- buy vs build, reuse
- change strategy - leave scope for changes in design, document thoughts on how to handle possible changes in design
- coding guidelines, conventions on naming, comments, formatting
Programming Environment
Spend time in selecting you language(s), tools, hardware and OS. Plan for both the present and future needs of your users and your developers -
- Will you users migrate to a different platform in the future e.g. from web to a mobile-app based interface?
- Should your developers write cross-platform code so that they can migrate to a different hardware/compiler/OS in the future?
- Where on the technology wave are the tools you intend to use i.e. are you using beta-quality tools and compilers and should expect to spend considerable time debugging or are you working on a mature set of tools and can find all the answers online?
Design
Todo - read No Silver Bullets by Fred Brook
This activity needs to be done for every project. For small projects it usually gets done as a conversation between developers at their desk. For large projects, its best to explicitly call this out, get everyone to review the architecture doc, assign subsystems to design and spend time in documenting the design, rationale behind it, alternatives and reason for not choosing them.
Good design characteristics -
- simple and easy to understand
- easy to maintain
- minimum number of connections between subsystems
- changes in one piece of the system shouldn't cause changes in others
- reusable
- portable
- high fan-in, large number of higher-classes make use of utility classes
- low fan-out, each higher-class makes use of a limited (low) number of lower-classes
- lean or minimalist design i.e. nothing more can be taken out of the system
- each level of decomposition provides a complete view and does not require dipping into the lower levels to gain an understanding its functionality
Possible approach to designing (distribute responsibility at each level) -
- start with a high level (single box) overview of the system
- break it down into 2 to 5 sub-systems, where each sub-system captures a significant portion of the functionality e.g. sub-system to read, write and manage the database, another sub-system to manage read the user's input and provide output to the user, etc
- further break down each of these sub-systems to smaller components e.g. database read component, database write component, database security component, user input screens, input validation component, user output screens, etc
- describe the various classes that will make up each component, the information they will contain (private), the information they will reveal (public), the classes they extend (inherit), mode of object instantiation (singleton, factory, other patterns)
- describe the routines that implement the functionalities for each of the classes
Different application types (mobile app vs low latency server process) and programming environments (c++ vs python) will require a change in the design approach.
Design For Test - to facilitate testing, design a component so that it can be tested separately from all other components that it needs to interact with.
Avoid Failure - its good to rely on experience of success with past projects/designs, but remember failures occur when designers do not carefully consider the ways in which a design can fail and instead copy-paste past experiences.
Design Practices
Iterate - multiple iterations over designing the system will eventually lead to a better more robust design. Its impossible to design the system in one attempt or to go from start to end sequentially. Keep refining the high level view based on the revelations from the low level design and vice versa.
Divide and Conquer - divide the large design problem into smaller pieces, repeat the division exercise till you reach a manageable problem size.
Top-down vs Bottoms-up - don't pick one approach over the other, utilize both. The top-down approach starts simple and builds towards more complex issues, whereas the bottoms-up approach starts with complexity and allows to move towards generalization. Both activities feed into each other.
Prototype - build small and simple prototypes to validate assumptions on a new technology or design approach. Iterative building prototypes for different parts of the system or the entire system is one way to identify risks and mitigate them early on in the project.
Collaborate - talk to your mates, find out what has worked for them on their projects, can you reuse their ideas?
Capture your design - in UML, formal design documents, on Confluence/Wiki, photographs, emails, code comments, whatever works.
Abstract Data Types
ADTs are collection of data and operations that work on the data. Benefits of ADTs -
- hide implementation details.
- a change to ADT does not affect the whole program i.e. changes need to be done in one place.
- collecting operations on a set of data in an ADT allows you to define an informative interface.
- improving performance can be done by focusing on the operations limited to an ADT.
- program is self documenting as you work with real world entities and not low level details e.g. insert car in parking lot instead of insert node in linked list.
Its intuitive to implement ADTs as classes in object oriented languages.
For non-OO languages ADTs can be implemented by defining an interface and explicitly isolating it from the rest of the code for e.g. in C a header file with exported functions act as an interface, the data type can be typedef-ed to represent the expected input/output.
Class Interface
An interface abstracts the implementation. A good abstraction offers routines that belong together. Guidelines on creating class interfaces -
- Present a consistent level of abstraction for e.g. a class representing a list of employees has an interface that allows adding or removing employees and then also allows iterating over the internal list of employees. This interface exposes the internal data structure for iteration but hides it when adding/removing employees.
- Provide routines that set and unset i.e. if there is method to enable something, then there must be a method to disable it. There may be corner cases where once a value is set it can't be reverted but that should be rare.
- Partition an interface into two or more classes when a set of routines work with a set of data and another set of routines work with another set of data and there is no overlap between these two.
- As classes are modified/evolve the interface needs to be looked at again if it needs to be modified, if the class needs to be split up or if it is still consistent and requires no change.
- Preserve the integrity of an interface each time there is a need to add a routine to it i.e. additions to the interface should be consistent with the initial design.
Encapsulation
A stronger concept than abstraction, is encapsulation i.e. hide the implementation details. Experienced developers will not differentiate between the two concepts as they are similar and should be enforced together. Guidelines on encapsulation -
- Hide more instead of less. When not sure if a routine should be public or if it is consistent with the interface, make it private.
- Don't expose member data in public.
- If the language permits, don't disclose private routines and data in the class header. This isn't always possible e.g. C++ headers do contain private declarations and allow users to examine private interface.
- Don't make assumptions about the users. A class interface should define its usage and so should the implementation of that interface. Any assumptions or special handling done for a subset of users violates this.
- Avoid friend classes. They violate the concept of encapsulation. There are rare situations where friend classes reduce complexity and their usage is generally accepted.
- A routine shouldn't be public if it uses other public routines.
- Don't disclose private implementation to users. For e.g. all public routines of a class might check if the public Init() function has been called and if not they could internally invoke it, users of this class could avoid calling Init() knowing that this is handled internally by the interface.
Containment
'has a' relationship. When a class contains another class object or native data type. For e.g. class vehicle has an engine, a set of tires, a steering wheel, etc.
Todo - refresh memory on private inheritance and 'has a' relationship
Inheritance
'is a' relationship. When one class specializes another class i.e. child class of a parent. For e.g. class FourWheelDrive is a specialization of class Vehicle.
The purpose of inheritance is to define a base class with common code and child classes with specialization code - to simplify the code base.
If a class is not intended for specialization make its members non-virtual in C++, final in Java or non-overridable in Visual Basic.
A good case for inheritance is when the child class objects can be referred to using the base class references and all routines of the base class imply the same meaning when using in each of the child classes. For e.g. base class Vehicle with derived classes FourWheelDrive and TwoWheelDrive, the routine Drive(int kms) has the same meaning when referring to objects of any type.
The interface and implementation inherited by the child class falls in one of these three -
- abstract routine i.e. inherit the routine's interface but not its implementation.
- overridable routine i.e. inherit the routine's interface and its implementation, further it can override the default implementation.
- non-overridable routine i.e. inherits both interface and its implementation, but can't override the default implementation.
A good designer will move common interface, data and behavior higher up in the inheritance tree. Abstraction will guide how high up in the tree these elements need to be.
A base class with a single derived class points to a design that anticipated future needs which haven't yet been realized. Its better to start of with a single class and bring in the parent-child relationship through a redesign at a later stage. Experienced designers will know which approach to take when.
When a derived class overrides a base class method to do nothing it points to a situation where the base class functionality was not designed properly. For e.g. base class Vehicle with method ClimbVertically() may be overridden to do nothing by class TwoWheelDrive.
Deep inheritance trees lead to code that is difficult to comprehend and maintain. Inheritance should be used to manage complexity and not to increase it.
Multiple Inheritance is useful when a class needs to display properties of multiple abstract types for e.g. a class may need to be both serilizable and sortable. In this case its better to inherit the interface from two different abstract types then to implement them without inheritance as that would appear as an addition of number of functions to a reader of the class.
Certain languages support inheriting multiple interfaces but not multiple implementations. C++ does not restrict inheriting multiple implementations.
Note - Inherit when you want the base class to control the interface, Contain when you want to control the interface.
A well prepared project -
- Defines the Problem without listing the solution
- Lists the Requirements explicitly i.e. scope, to avoid confusion or arguments at a later stage
Requirements
Requirements will change as the project progresses. The cost of changing them and the rationale behind the change should be discussed with everyone. Plan for changes, put in place a development approach that accommodates them e.g. iteratively refine a prototype into a final product. Manage the changes, don't let them ruin the project and don't guard the project so that you end up building a useless product.
Know when to dump the project - has competition made an unexpected move? has there been a technology change? economic/financial surprise? force majeure?
Checklist of questions for gathering requirements
- Functional - inputs, outputs, interfaces, user tasks
- Non-Functional - response time, throughput, memory and storage limits, maintainability
- Quality - document requirements in users language, conflict amongst requirements, level of detail, likelihood of change in a requirement
- Completeness - areas of incomplete requirements, level of comfort with the requirements
System Architecture
The step between requirements and construction. It can specify both system wide design constraints or those that apply to subsystem. The level of detail may wary. The intent is that Architecture Document provides the overall guidance to developers, allows them to partition the work, work independently and also coherently.
Broadly it covers these -
- various alternatives considered and the reason for selection one program organization over others.
- the building blocks of the system, their interfaces and communication rules.
- file formats, database, performance tweaks to access data containers.
- user experience (XD), inputs and outputs, security, managing resources, performance and scalability, internationalization and localization.
- defines how to handle errors, detection and propagation, input validation and assumptions.
- reliability of the system, techniques to backup to last good state, auxillary processes, hot vs cold standby's.
- over-engineer for superb performance versus simple solution to get it to work
- buy vs build, reuse
- change strategy - leave scope for changes in design, document thoughts on how to handle possible changes in design
- coding guidelines, conventions on naming, comments, formatting
Programming Environment
Spend time in selecting you language(s), tools, hardware and OS. Plan for both the present and future needs of your users and your developers -
- Will you users migrate to a different platform in the future e.g. from web to a mobile-app based interface?
- Should your developers write cross-platform code so that they can migrate to a different hardware/compiler/OS in the future?
- Where on the technology wave are the tools you intend to use i.e. are you using beta-quality tools and compilers and should expect to spend considerable time debugging or are you working on a mature set of tools and can find all the answers online?
Design
Todo - read No Silver Bullets by Fred Brook
This activity needs to be done for every project. For small projects it usually gets done as a conversation between developers at their desk. For large projects, its best to explicitly call this out, get everyone to review the architecture doc, assign subsystems to design and spend time in documenting the design, rationale behind it, alternatives and reason for not choosing them.
Good design characteristics -
- simple and easy to understand
- easy to maintain
- minimum number of connections between subsystems
- changes in one piece of the system shouldn't cause changes in others
- reusable
- portable
- high fan-in, large number of higher-classes make use of utility classes
- low fan-out, each higher-class makes use of a limited (low) number of lower-classes
- lean or minimalist design i.e. nothing more can be taken out of the system
- each level of decomposition provides a complete view and does not require dipping into the lower levels to gain an understanding its functionality
Possible approach to designing (distribute responsibility at each level) -
- start with a high level (single box) overview of the system
- break it down into 2 to 5 sub-systems, where each sub-system captures a significant portion of the functionality e.g. sub-system to read, write and manage the database, another sub-system to manage read the user's input and provide output to the user, etc
- further break down each of these sub-systems to smaller components e.g. database read component, database write component, database security component, user input screens, input validation component, user output screens, etc
- describe the various classes that will make up each component, the information they will contain (private), the information they will reveal (public), the classes they extend (inherit), mode of object instantiation (singleton, factory, other patterns)
- describe the routines that implement the functionalities for each of the classes
Different application types (mobile app vs low latency server process) and programming environments (c++ vs python) will require a change in the design approach.
Design For Test - to facilitate testing, design a component so that it can be tested separately from all other components that it needs to interact with.
Avoid Failure - its good to rely on experience of success with past projects/designs, but remember failures occur when designers do not carefully consider the ways in which a design can fail and instead copy-paste past experiences.
Design Practices
Iterate - multiple iterations over designing the system will eventually lead to a better more robust design. Its impossible to design the system in one attempt or to go from start to end sequentially. Keep refining the high level view based on the revelations from the low level design and vice versa.
Divide and Conquer - divide the large design problem into smaller pieces, repeat the division exercise till you reach a manageable problem size.
Top-down vs Bottoms-up - don't pick one approach over the other, utilize both. The top-down approach starts simple and builds towards more complex issues, whereas the bottoms-up approach starts with complexity and allows to move towards generalization. Both activities feed into each other.
Prototype - build small and simple prototypes to validate assumptions on a new technology or design approach. Iterative building prototypes for different parts of the system or the entire system is one way to identify risks and mitigate them early on in the project.
Collaborate - talk to your mates, find out what has worked for them on their projects, can you reuse their ideas?
Capture your design - in UML, formal design documents, on Confluence/Wiki, photographs, emails, code comments, whatever works.
Abstract Data Types
ADTs are collection of data and operations that work on the data. Benefits of ADTs -
- hide implementation details.
- a change to ADT does not affect the whole program i.e. changes need to be done in one place.
- collecting operations on a set of data in an ADT allows you to define an informative interface.
- improving performance can be done by focusing on the operations limited to an ADT.
- program is self documenting as you work with real world entities and not low level details e.g. insert car in parking lot instead of insert node in linked list.
Its intuitive to implement ADTs as classes in object oriented languages.
For non-OO languages ADTs can be implemented by defining an interface and explicitly isolating it from the rest of the code for e.g. in C a header file with exported functions act as an interface, the data type can be typedef-ed to represent the expected input/output.
Class Interface
An interface abstracts the implementation. A good abstraction offers routines that belong together. Guidelines on creating class interfaces -
- Present a consistent level of abstraction for e.g. a class representing a list of employees has an interface that allows adding or removing employees and then also allows iterating over the internal list of employees. This interface exposes the internal data structure for iteration but hides it when adding/removing employees.
- Provide routines that set and unset i.e. if there is method to enable something, then there must be a method to disable it. There may be corner cases where once a value is set it can't be reverted but that should be rare.
- Partition an interface into two or more classes when a set of routines work with a set of data and another set of routines work with another set of data and there is no overlap between these two.
- As classes are modified/evolve the interface needs to be looked at again if it needs to be modified, if the class needs to be split up or if it is still consistent and requires no change.
- Preserve the integrity of an interface each time there is a need to add a routine to it i.e. additions to the interface should be consistent with the initial design.
Encapsulation
A stronger concept than abstraction, is encapsulation i.e. hide the implementation details. Experienced developers will not differentiate between the two concepts as they are similar and should be enforced together. Guidelines on encapsulation -
- Hide more instead of less. When not sure if a routine should be public or if it is consistent with the interface, make it private.
- Don't expose member data in public.
- If the language permits, don't disclose private routines and data in the class header. This isn't always possible e.g. C++ headers do contain private declarations and allow users to examine private interface.
- Don't make assumptions about the users. A class interface should define its usage and so should the implementation of that interface. Any assumptions or special handling done for a subset of users violates this.
- Avoid friend classes. They violate the concept of encapsulation. There are rare situations where friend classes reduce complexity and their usage is generally accepted.
- A routine shouldn't be public if it uses other public routines.
- Don't disclose private implementation to users. For e.g. all public routines of a class might check if the public Init() function has been called and if not they could internally invoke it, users of this class could avoid calling Init() knowing that this is handled internally by the interface.
Containment
'has a' relationship. When a class contains another class object or native data type. For e.g. class vehicle has an engine, a set of tires, a steering wheel, etc.
Todo - refresh memory on private inheritance and 'has a' relationship
Inheritance
'is a' relationship. When one class specializes another class i.e. child class of a parent. For e.g. class FourWheelDrive is a specialization of class Vehicle.
The purpose of inheritance is to define a base class with common code and child classes with specialization code - to simplify the code base.
If a class is not intended for specialization make its members non-virtual in C++, final in Java or non-overridable in Visual Basic.
A good case for inheritance is when the child class objects can be referred to using the base class references and all routines of the base class imply the same meaning when using in each of the child classes. For e.g. base class Vehicle with derived classes FourWheelDrive and TwoWheelDrive, the routine Drive(int kms) has the same meaning when referring to objects of any type.
The interface and implementation inherited by the child class falls in one of these three -
- abstract routine i.e. inherit the routine's interface but not its implementation.
- overridable routine i.e. inherit the routine's interface and its implementation, further it can override the default implementation.
- non-overridable routine i.e. inherits both interface and its implementation, but can't override the default implementation.
A good designer will move common interface, data and behavior higher up in the inheritance tree. Abstraction will guide how high up in the tree these elements need to be.
A base class with a single derived class points to a design that anticipated future needs which haven't yet been realized. Its better to start of with a single class and bring in the parent-child relationship through a redesign at a later stage. Experienced designers will know which approach to take when.
When a derived class overrides a base class method to do nothing it points to a situation where the base class functionality was not designed properly. For e.g. base class Vehicle with method ClimbVertically() may be overridden to do nothing by class TwoWheelDrive.
Deep inheritance trees lead to code that is difficult to comprehend and maintain. Inheritance should be used to manage complexity and not to increase it.
Multiple Inheritance is useful when a class needs to display properties of multiple abstract types for e.g. a class may need to be both serilizable and sortable. In this case its better to inherit the interface from two different abstract types then to implement them without inheritance as that would appear as an addition of number of functions to a reader of the class.
Certain languages support inheriting multiple interfaces but not multiple implementations. C++ does not restrict inheriting multiple implementations.
Note - Inherit when you want the base class to control the interface, Contain when you want to control the interface.
