What I am thinking about is if we can define metrics on on the one hand to calculate complexity and on the other hand patterns that arise when complexity becomes more complex. We could use this to more effectively use the right tools and processes on the right level.
If you have no clue of what I am talking about … I have the problem that probably you will not have a clue even when I explain it…. but I will try (briefly):
If you have 10 stamps you would care less about a system to manage stamps and even less about a process of ordening stamps but you are interested in their names e.g. “10 cent christmas stamp”, it’s what children do: they slowly begin to recognize names with objects: by doing so they start managing the complexity around them.
If you get to 10.000 stamps you slowly begin to feel that you: 1. want to orden them, e.g. by country or by value or by “pretty and not pretty” this is more or less what science does : this is of the family of apes and this is of the family of trees, these are called taxonomies. 2. you would like to have e.g. spreadsheet to make a list of them so you can go to a stamp collectors market and trade based on this list. In this list you would define properties of the stamp alongside with the name, this is what database does: being a structured collection of data with the emphasis on structured and what that means. You don’t need a process you just print out your list and go to these collector markets.
If you get to 1.000.000 stamps in your collection you definitely feel the need for more taxonomies (e.g. “tags”, “categories”), a complexer database and many more ways to handle this complexity. You also find the urge to slowly introduce some processes and stamp rules: stamps of high value will go in the high value album so you not mistakenly trade them etc… etc… You also will have the need of albums which give indications of default ordenings of stamps and catalogues which give the average values and so on. A finite list of possbilities to manage complexity. (I would like to make such a list when I have time, it would be needed for the solution of all of this)
Now… suppose you have developed a system or a novice process (catalogue, description, templates for entering stamp information or sell stamp forms, people) to manage large collections of stamps. Would you try to sell these to persons with only 10 stamps? Probably not.
Now… suppose you own a stamp company that has 99.000 departments in 200 countries. Managing this is uber-complex. You feel the need for standaard tools, databases, processes and so on. But for a specific department in one specific country there could be just 1 person managing 10 stamps. This person could care less about a system managing stamps and he or she is probably right because it would take more time to work with this system or process than simply handle the 10 stamps.
Now… IF that stamp company would pre-scribe 1 system to manage stamps the smaller departments would not benefit from it since their compleity level is not high enough. But… IF a stamp collection system would exist that would “scale” tools and processes depending on the complexity of that department … that would work.
So that is what I am thinking about: a scalable system that, depending on complexity, introduces or scales complexity reduction tools and processs automatically. That “automatically” means no human intervention would be needed since the system would know that with a metric of 5 stamps no database is needed and with 10.000 stamps it would automatically introuduce new “help” since the break even point between “going crazy of too many stamps” and “learning and registering them in a new tool” would be ideal.
Another example is development:
If you are a single person out there creating your own website with WordPress … you could care less about version management. You don’t even have a clue what it is.
If you have 25 websites that you work on in parallel you read about version management and e.g. install subversion or git or mercurial to at least have components of often used chunks of code that you can name e.g. “jquery gallery version 0.1″ and “jquery gallery verison 0.2″ that you use in different sites at the same time (version management tools give you the benefit that you can name each iteration of your code). At that stage you could care less about complexer management using e.g. configuration management tools and processes. you probably will work with “todo lists” in a shared spreadsheet or something alike. Just because the complexity level does not yet drive you crazy enough. “you can handle it”.
If you have 100 large websites and you work on them with 100 people in bursts of improvement… you slowly go crazy because people are doing the same stuff on the same code and you feel the need to more componentize stuff, you might even feel the need to somehow handle the environment you develop it on and the environment you deploy to, so you slowly move to configuration management and configuration management tooling e.g. ClearCase: being able to “play” with bigger chunks of versions that you combine e.g. “on production server 23″ and also under development on development server 2″. You start using some kind of system that can handle “what everyone is doing”: a change managment system. It’s just because the complexity is there … so you are forced to use “something” that helps you reduce or at least manage that complexity.
At a certain stage e.g. working with 500 people on many different parts of code you even would hire “people who manage that complexity” e.g. configuration managers or change managers or requirement managers whose job it basically is to think about ways to reduce complexity or at least make it manageable. A good configuration manager has experience on how to reduce complexity. A really hard job (and ofcourse not understandable why it would needed if you work in a less complexer environment or are just 1 node in that group of 500 people versus being the manager of those 500 people).
So… what I am thinking about is … if some kind of system could be made that calculates complexity and based on that automatically “surrounds” people with the appropriate level of tools to manage complexity that would be … nice.
The example above are just specific examples of tools or processes around stamps or “IT” what Im thinking of is a bigger approach, start here: http://en.wikipedia.org/wiki/Ontology. And read: http://en.wikipedia.org/wiki/Upper_ontology and maybe the chapter “Arguments for the infeasibility of an upper ontology” and “Arguments for the feasibility of an upper ontology“. Because in the end, it would be needed to create something like this on a foundational layer as one of the chunks “in there”.
In the IT world some ready made ontologies are available so that the persons responsible for thinking of things to manage complexity do not have to start off from scratch. You can think of all kinds of frameworks ranging from ITIL to RUP that encompass assets and processes and handy ready made templates to “measurement of maturity” as CMMI, a gazillion amount of tools that integrate or integrate not with these processes and so on. And then on a meta level methods and tools to combine these frameworks and produce new ones. Actually … too much to descibe in a quick blog post.
So… I think… there is a little puzzle piece missing here. It’s the “component” that scales the meta solution more or less automagically based on complexity level.
And I do realize that metrics for complexity are dependent on many factors including the specific domain.
Noone will ever read this so I´m safe So… let me copy and paste something to combine in this to combine later with upper ontologies and meta modeling and “the scale on when what is needed for the people in the specific case based on the complexity surrounding them” :
Process philosophy (or ontology of becoming) identifies metaphysical reality with change and development. Since the time of Plato and Aristotle, philosophers have posited true reality as “timeless”, based on permanent substances, whilst processes are denied or subordinated to timeless substances. If Socrates changes, becoming sick, Socrates is still the same (the substance of Socrates being the same), and change (his sickness) only glides over his substance: change is accidental, whereas the substance is essential. Therefore, classic ontology denies any full reality to change, which is conceived as only accidental and not essential. This classical ontology is what made knowledge and a theory of knowledge possible, as it was thought that a science of something in becoming was an impossible feat to achieve.
In opposition to the classical model of change as purely accidental and illusory (as by Aristotle), process philosophy regards change as the cornerstone of reality–the cornerstone of the Being thought as Becoming. Modern philosophers who appeal to process rather than substance include Heidegger, Charles Peirce, Alfred North Whitehead, Robert M. Pirsig, Charles Hartshorne, Arran Gare and Nicholas Rescher. In physics Ilya Prigogine distinguishes between the “physics of being” and the “physics of becoming”. Process philosophy covers not just scientific intuitions and experiences, but can be used as a conceptual bridge to facilitate discussions among religion, philosophy, and science
Will I go crazy thinking about this?
A single developer who starts using a version management system is doing this to manage complexity. His concern is “his code” and he started doing this because he started thinking about the complexity of his parallel used codebases. a project manager who starts to chunk up work in a breakdown structure is doing exactly the same. We can scale this larger and larger and larger and larger and at the end we end up with branches of philosophy. whose concern it is to manage complexity of “everything” via a structured approach. On this highest level the reason to do this is to “explain the world” and practically it gave us all sciences and probably everything you around you. So if all this can be made scalable it has to work on the top level and the lowest level and all the levels in between and depending on the level the complexity management and reduction meta level would scale along. Basically everyone human is doing this stuff. In your household you manage “budget” by making a taxonomy of companies and organizations you have a financial relation with and quantify this by adding “fields” such as amount of money that transferred between you and the company. (You manage “likes” on facebook by giving this same taxonomy of companies and organization a “like” or “no like”). If you run a smaller project you will use simpler tools and processes to manage the complexity. If you are responsible of multiple smaller projects that use these simpler tools and processes you will use tools and processes that are available to manage even greater complexity etc… but the “meta model” behind ALL of these tools, processes scaling up would be needed to make such a component.
Maybe a key lies in the philosophy of science, but I have only started reading books about it.