Solving problems by adding an abstraction [tech]
The technique that I’ve used most often to solve programming problems is adding a new level of abstraction. In fact this method is called the fundamental theorem of software engineering.
We can solve any problem by introducing an extra level of indirection
Indirection above is analogous to abstraction.
Usually when you start building an application (or library) requirements are simple. At some point there are some additional requirements that require adding a new abstraction. For example you could have a dictionary app that supports a single language. If you want to add support for multiple languages with reuse, then the language needs to be configurable. If there are some hard coded references to the parameter, those need to be made variables and externally configurable. After that it is possible to run the application in a different configuration.
By itself creating new abstractions seems simple enough. When a new abstraction is added, finer control of the sub-systems is either lost or harder to control. Adding abstractions is usually much easier when the change does not affect the public or exposed API. For changes that affect clients that cannot be changed, a compatibility API is needed.
An alternative approach to adding abstraction based on requirements would be to go ahead and build some abstraction during early design. This works best if the problem domain is well understood. Otherwise adding abstractions before being used adds complexity without value and it's likely that future requirements necessitate a different solution.
Over the lifetime of long running projects, multiple abstractions and extensions are added. If the requirements and abstractions are added in an ad hoc manner without a vision for the larger system, then the project can lose focus. Managing technical debt and change becomes a larger cost than adding new functionality.
Eventually it's best to decompose to smaller units of functionality. Refactoring a system to extract sub-systems that are stable and reusable is a good approach. I prefer using the term system since it can be used to describe small or large scope from methods or classes to entire micro-services and beyond.
In closing, well architected solutions provide good abstractions to solve hard problems in an easier manner providing much more value compared to the fine grained control that is lost.