|
Reading this fantastic book,
Semantic Software Design - O'Reilly pub - A New Theory and Practical Guide for Modern Architects*[^], where the author suggests that Software Architect is a failed analogy & should never be used. He goes back thru history of software development and explains where this term arose & that it was never meant to truly explain the work we do in designing systems.
Instead, the author likens Software Development & Design to the realm of Artistic Endeavour.
He also explains that thinking in this way was supposed to get us to repeatable processes that would make software more reliable like any other commodity, but it never has.
*Also, he only uses the word Architect in the sub-title to get the attention of people who think they are software architects.
Quote: Why Software Projects Fail
As I mentioned earlier in this chapter, software projects fail at an astonishing rate:
In 2008, IBM reported that 60% of IT projects fail. In 2009, ZDNet reported that 68% of software projects fail.
By 2018, Information Age reported that number had worsened to 71% of software projects being considered failures.
Deloitte characterized our failure rate as “appalling.” It warns that 75% of Enterprise Resource Planning projects fail, and Smart Insights reveals that 84% of digital transformation projects fail. In 2017 Tech Republic reported that big data projects fail 85% of the time.
According to McKinsey, 17% of the time, IT projects go so badly that they threaten the company’s very existence.
Numbers like this rank our success rate somewhere worse than meteorologists and fortune tellers.
Have any of you read this book? What do you think about this?
Here's some additional notes from the book that explain the author's point even better.
Quote: A McKinsey study of 5,600 companies found the following:
On average, large IT projects run 45 percent over budget and 7 percent over time, while delivering 56 percent less value than predicted. Software projects run the highest risk of cost and schedule overruns.
Of course, some projects come in on time and on budget. But these are likely the “IT” projects cited in the McKinsey study, which include things like datacenter moves, lift-and-shift projects, disaster-recovery site deployments, and so forth. These are complex projects (sort of: mostly they’re just big). I have certainly been involved in several such projects. They’re different than trying to conceive of a new software system that would be the object of design.
The obvious difference is that the “IT” projects are about working with actual physical materials and things: servers to move and cables to plug in. It’s decidable and clear when you’re done and what precise percentage of cables you have left to plug in. That is, many CIO IT projects of the back-office variety are not much more creative than moving your house or loading and unloading packages at a warehouse: just make the right lists and tick through them at the right time.
It’s the CTO’s software projects that run the greatest risk and that fail most spectacularly. That’s because they require the most creative, conceptual work. They demand making a representation of the world. When you do this, you become involved in signs, in language, in the meaning of things, and how things relate. You’re stating a philosophical point of view based in your epistemology.
You’re inventing the context wherein you can posit signs that make sense together and form a representation of the real world.
That’s so much harder.
|
|
|
|
|
|
You ask for it, but what is your opinion?
I think: Yes, an architect is needed to tell the staff what has to be build 
|
|
|
|
|
I used to be a software architect, and I always felt the title was something of a misnomer. I was basically an in-house consultant and overarching team lead, for lack of a better way to put it. I still don't know exactly what a software architect is supposed to be. I used to think I did, until I did it. Frankly, I thought there would be a lot more UML involved, but I'm glad there wasn't.
As far as my job responsibilities, this was at more than one company, so it wasn't just one company's quirks.
Real programmers use butterflies
|
|
|
|
|
honey the codewitch wrote: I thought there would be a lot more UML involved, but I'm glad there wasn't
Ah UML. Another one of those things that is _fantastic_ & _amazing_ when used properly and _disgusting_ and _painful_ when used improperly.
I have used a UML diagram to communicate one specific point very quickly to an audience & it was amazing.
It got people to take action & understand system boundaries and more. It was such an amazing experience.
However, I've also been slammed over the head with UML tomes that are just a large brick of documentation that are really just for show purposes and it was terrible. 
|
|
|
|
|
Likening software design to artistic endeavor is even more misleading than likening it to architecture. Architecture also has an artistic aspect, but it is also constrained by real-world feasibility and what the customer wants.
The folks who first documented software patterns were very influenced by the writings of Christopher Alexander, an architect who authored The Timeless Way of Building and A Pattern Language. But software had already adopted the term architect at that point.
Fred Brooks (The Mythical Man-Month, 1975) used architect to mean the person who specified a software system's capabilities and behavior, not its design. But since then, it has come to mean more of the latter.
Using architect is in some ways misleading, but does it matter if software architect has a generally agreed upon meaning? I'd also say that engineer is also somewhat misleading, but does it matter if software engineer also has a generally agreed upon meaning?
As far as failed software projects go, don't get me started. In my opinion, the biggest contributing factor is that hardly anyone sees their company as being in a software business. They typically view the software team as merely a cost center, and software as a necessary evil that is simply a factor input to an end product. When the importance of software excellence is poorly understood and rewarded, the company fails to develop or retain software architects and ends up paying the price.
|
|
|
|
|
Very good & interesting points.
Greg Utas wrote: The folks who first documented software patterns were very influenced by the writings of Christopher Alexander
The author mentions this exact book in this first chapter and takes it all the way back to 1968 NATO meeting... (underlined portion denotes the quote by Peter Naur at the 1968 meeting).
This is basically the first recorded occurrence of thinking of softwar devs as Architects.
Quote: The term “architect” as used in software was not popularized until the early 1990s. Perhaps the first suggestion that there would be anything for software practitioners to learn from architects came in that NATO Software Engineering conference in Germany in 1968, from Peter Naur:
Software designers are in a similar position to architects and civil engineers, particularly those concerned with the design of large heterogeneous constructions, such as towns and industrial plants. It therefore seems natural that we should turn to these subjects for ideas about how to attack the design problem. As one single example of such a source of ideas, I would like to mention: Christopher Alexander: Notes on the Synthesis of Form (Harvard Univ. Press, 1964) (emphasis mine).
This, and other statements from the elder statesmen of our field at this conference in 1968, are the progenitors of how we thought we should think about software design. The problem with Naur’s statement is obvious: it’s simply false. It’s also unsupported. To state that we’re in a “similar position to architects” has no more bearing logically, or truthfully, to stating that we’re in a similar position to, say, philosophy professors, or writers, or aviators, or bureaucrats, or rugby players, or bunnies, or ponies. An argument by analogy is always false. Here, no argument is even given. Yet here this idea took hold, the participants returning to their native lands around the world, writing and teaching and mentoring for decades, shaping our entire field. This now haunts and silently shapes—perhaps even circumscribes and mentally constrains, however artificially—how we conduct our work, how we think about it, what we “know” we do.
Greg Utas wrote: When the importance of software excellence is poorly understood and rewarded, the company fails to develop or retain software architects and ends up paying the price.
Yes, I've seen this too. I work on legacy systems which were written by novices who didn't even know how to create a database with keys. They literally created string keys for every table -- ugh!
Companies who don't know to pay for properly experienced people to write software definitely do damage that they pay 10X as much for -- and more.
Here's the great quote:
Red Adair said If you think it's expensive to hire a professional to do the job, wait until you hire an amateur.
|
|
|
|
|
In the Naur quote, it's dubious to claim that we should look to architecture for answers on how to attack software design. On the other hand, using Alexander's format to document patterns is quite reasonable, though it can get verbose.
I'm curious why the author harps on the use of architect. He says it "haunts" how we conduct our work, so he must have some opinions on how our work would be better conducted.
|
|
|
|
|
Greg Utas wrote: I'm curious why the author harps on the use of architect. He says it "haunts" how we conduct our work, so he must have some opinions on how our work would be better conducted.
Yes, he does have an opinion about it and that is really what the book is all about.
I'm about to start chapter 2 to get a better idea of what his frame of thinking involves.
Also, I think he is saying that using the idea of Architect has limited things in many ways.
People often get stuck in frames of thinking when they universally apply a term.
He's saying that the term Architect has led people down paths of thinking that has gotten them stuck in dead ends.
Now, many people (such as yourself it sounds) have understood that this metaphor (architect) only carries so far and then as a Real Practitioner (person who has written software to solve problems) often understand the nuances of how Architect does apply & doesn't and have overcome all terms because as Practitioners who actually have to produce a result we don't just get stuck on terms : we get the job done.
Some of the process of Real Software Development may be ineffable -- such as the way that Art sometimes is. You can't explain it, but you know it when you see it.
I'm reading the book not necessarily because I agree with him 100% but because I like to see all the nuanced explanations from all sides. I try to take what I like and throw the rest out.
|
|
|
|
|
Thanks for posting all of this. I'd be interested in your take on the book as you read through it. If it's full of shite, I'm happy to let someone else spend their time on it. 
|
|
|
|
|
I'll read chapter 2 and try to post something explanatory yet succinct enough.
Its a long chapter.
|
|
|
|
|
Chapter 2 was extremely long and the author really went into the weeds.
Main Point Is What We All Already Know
There were some interesting things but for the most part what you discover -- as we who have been building software for companies for years have already discovered -- is that there is no One Process to developing Software. There are instead numerous ways to get to the same finish line. Whichever one works best for you & your team is the best.
Here are a few snippets from the book that show how far the author gets out into the weeds.
Quote: The Myth of Requirements
In system design, we speak of the “requirements.” This word creates a false center, a supposed constant, which creates problems for our field. These problems come in the form of a binary opposition set up between the product management team and the development team. It supposes, in the extreme form, that the product management team knows what to build and that the development team are passive receptacles into whom we insert this list of what they are required to build. Within an Agile method, some freedom is perhaps allowed to the development team in how to design within that list of requirements.
The requirements, however, do not exist. But the requirements, like everything else of value, are just made up by someone. They are not first known and then told. They are invented.
He compares this idea of inventing requirements to the act of creating a character in a story.
Quote: How do we know that Indiana Jones is the archaeology professor who finds the Lost Ark of the Covenant? Because George Lucas invented a character named Indiana Smith, and Steven Spielberg didn’t like the name so he changed it to “Indiana Jones.” And all of a sudden there is a world of the 1930s and a man standing in it and he needs to go do something and someone needs to get in his way and how might that work? That’s how requirements are made, in the movies and in software. People make stuff up.
When you make stuff up as a software designer, that world, like the world of the movie into which you posit a character with a conflict, is your context. It’s the place where you posit signs that have meaning in relation to one another. It’s your semantic field.
Quote: Semantics, as a field, is concerned with the production of meaning, and how logic and language are used.
Semantics = logic + language
This is almost the "Jerry Maguire Situation" from the author. In a night of passionate turmoil he makes meaning of everything and comes to the conclusion "We should all be nice to each other."
Oh, yes, shouldn't we.
In the case of this author, he is like, "Everything is language. Projects fail because we all have our own brains and all of us understand what we are building in different ways."
Oh, yes, that's right isn't it. but you can't quite fix that. Sheesh.
Here's a final quote from chapter 2.
Quote: The problem with software—a chief reason our projects fail—is a failure of our language. We are not architects. Not even close. We do not build buildings with an obvious and known prior purpose, which is an approximate copy of the same kind of building people have been making and using for thousands of years, using tangible commodity materials on a factory line. Quite the opposite.
Our only material is that of language and ideas, names and meanings, signifiers and signifieds. Our only material is semantics.
So, the author is off the rails really. He found something that worked for him and which he can use to drive teams and their belief system to get things done well. But can it translate to other teams?
Probably. But probably because any passionate person with enough company-power and political-power can make almost any plan work.
Again, the point of it all is:
Find a process that works for you and your team. That will change as time changes and people change and the company changes.
|
|
|
|
|
Thanks for the summary. I definitely agree that there isn't One Process for developing software.
Quote: But the requirements, like everything else of value, are just made up by someone. They are not first known and then told. They are invented.
Sure, but what does this mean for software design? The customer may have requirements that are impossible or difficult to change, though we can suggest improvements. In other cases, requirements come from standards, so we need representatives in standards fora with whom we can work to steer the requirements in a direction that is advantageous or at least sensible.
Quote: The problem with software—a chief reason our projects fail—is a failure of our language.
This overstates the case. But I've been in design meetings where people are in vigorous disagreement, only to eventually realize that they were actually on the same page. Patterns are useful because they provide a consistent terminology that markedly reduces such misunderstandings. But the misunderstandings in themselves don't guarantee failure, and avoiding them doesn't guarantee success.
Quote: We are not architects. Not even close. We do not build buildings with an obvious and known prior purpose, which is an approximate copy of the same kind of building people have been making and using for thousands of years, using tangible commodity materials on a factory line. Quite the opposite.
This analogy works when comparing a building to an entire system. Unlike a building, it costs very little to replicate software, so if there's already software that does pretty much what you need, you can reuse it instead of rebuilding it. But it's not a good analogy when comparing a building to the components and strategies involved in developing a system. Systems within the same domain (web servers, editors, SCADA, financial transactions, avionics, telecom...) can reuse components and techniques, which can be documented by patterns that help to identify the appropriate ones. Even if code reuse isn't possible, design reuse often is, and we gain a lot even if that's the commodity.
|
|
|
|
|
I may not be the best person to comment on this but having read some of this thread I'd like to share my 50 cents on the issue.
I may say I'm a real software architect for the reason that I am both a software developer (20+ years) and an architect (3 year) and one thing drew my attention from the start of my second undergrad course: how there are, indeed, similarities between both. I'd like to comment on that and why I believe the title of software architect should yes exist.
Architecture is not only design, as most suggest, but also about foundation and structure and communication, about finding out what the needs our clients not even know they have and deal with the multiple interests of people in a building. That is what happens when we deal with people: they have their conflicts (not only of interest in the project but in between themselves), they change their minds all the time, from the first sketch to often when the building is already "finished" (whoever said a building is a finished unmodifiable thing has never had to reform/repurpose). Also, my professors often said an architect is more of a shrink than a designer or engineer. Does that sound familiar?
However, architecture is a discipline that has a history of more than 4,000 years compared to software development. Sure, there are practices well proven in it but still there are plenty on the open, methods, tools and materials are not static and are being developed and studied everyday. We also work with projects (and PMBOK could also be applied to it but rarely is, as far as I know because architects, at least here in Brazil, have not discovered it yet) but have consolidated means of documenting a project so other workers can "implement" it. Also, at least here in Brazil, an architect is responsible for conducting and overseeing a building construction, although the architect creating the project not always conducts it construction).
In my point of view, no one is wrong about what they said: requirements are just invented by someone but they do exist; there is more than one way of doing things and none are wrong, just some work better for some people while not for others; and the biggest problem we face is language, communicating intents, ideas, concepts. And this works both for software and architecture. Projects fail everywhere because of communication, because it is hard for the non-professional person to understand the effort it takes to elaborate a project, develop it and the value it would aggregate. Architects have been working on making people understand their value and the value of having an architectural project for years now so I don't expect non-IT people to understand it for software in my lifetime. Architecture is all about communication.
IMHO, one of the things that could benefit of this discussion would be clearly defining the role of the software architect. If we are to maintain the analogy, however, having a software architect closely related to the role of an architect could mean trimming some other roles we currently use (which would also be good because it seems every day a new role pops up from nowhere), e.g. product owners and project managers, since these are roles of the architect. The role of the architect is understanding what clients need (even when them themselves don't know), detail what needs to be clearly understood (either by those who will build it or by the client, and in software it would be much less than in a building, thrust me), and coordinate the efforts of the team to fulfil those needs. In software development, we divide those responsibilities among many people but I still cannot say which way works best.
We could also benefit both architects and software architects with more soft skills and psychology studies (I had nearly none on both) but there are many things we do right in software development too, like working with prototypes and UML. Here in Brazil, architecture prototypes translate to 3D renderings (I've heard in other countries a hand sketch is still more common) but don't go thinking people understand that is not the building itself; they still expect to move in the week after the presentation. UML might be far from perfect and far from being the same as a detailed architectural project (which can go way over 100 pages easily) and I do resort to using UML only when I believe some implementation detail might not be clearly understandable by developers. Clients also change their minds about the project all the time, even while it is already being build (yes, buildings too), we all just have to have the means to accomodate those changes, to understand the change is inevitable. But I also remember one of my professors (in software metrics) presenting us with a spreadsheet for a construction day one saying this was the dream for software engineering though I also understand clearly now how difficult (if not impossible) it is to achieve that level of accuracy for software development.
I know in the end this got way longer than I expected and I still haven't managed to read on the history behind the analogy (but I will, I just put it on my reading list) but I believe there is a correlation between architecture and software development and it just may be stronger than most think. I've recently discovered architects here in Brazil have discovered and are experimenting with agile methods in architectural project so it may be possible more of the methods and practices we develop and use for software may make their way into architecture itself in the future. Will it work? Time will tell.
- Leonardo
|
|
|
|
|
Great post.
Leonardo Pessoa wrote: Architecture is not only design, as most suggest, but also about foundation and structure and communication, about finding out what the needs our clients not even know they have and deal with the multiple interests of people in a building
Leonardo Pessoa wrote: Architecture is all about communication.
Those two things are really the foundation of it. Along with psychology, as you said.
|
|
|
|
|
Communication skills are definitely important for an architect. So is defining the role, but it must be tailored to the setting. For example, the products that I worked on were usually constrained by standards. I therefore wasn't involved in requirements, though I read reports from people who attended standards meetings and offered suggestions. I would use product architect for a role that focuses on a product's requirements and software architect for a role that focuses on how it will be implemented. In some cases, the roles might be merged.
Our requirements were complex, so we needed application frameworks. I'd seen examples of architects who didn't code, with the result that their designs failed to get implemented properly. I therefore also implemented frameworks. Their code was the only object model that application developers needed, so we used UML diagrams primarily for training purposes, to document frameworks once they had become fairly stable.
|
|
|
|
|
But is this realization not a step further already? Many times I personally think that more of these epiphanies should be shared, as our own frame of thought guides and limits us immensely. Being made aware that something might be hindering our communication could already be enough to push us to notice and improve upon it.
|
|
|
|
|
It exists merely to distinct between a junior.
Bastard Programmer from Hell 
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
|
|
|
|
|
Eddy Vluggen wrote: It exists merely to distinct between a junior.
You are correct. But it is too bad (and probably a part of the entire problem) that there isn't a good measurable way to distinguish between a junior and another role.
A lot of this is laziness on companies' part. They never attempt to describe why someone is an "architect" versus a junior.
I worked for a guy who was an architect at a large mortgage bank.
I was a lowly Software Developer and had to use an API that he created to wrap some things up in C#.
I found some problems and started asking him why he did it that way. I explained how it failed and that he was using the underlying C# API actually incorrectly.
He said, "I copy-pasted that code from somewhere else." He never tested it, of course, because An Architect doesn't do that!!! 
At this point I had 10 years of Software Dev experience and had started using C# & .NET when it pre-released (2000 or so). I had been using C# for 5 years at this point.
I asked him, "how long have you been using C#?"
Architect : Oh about 6 months
Me: How long have you been an Architect on the project?
Architect: I used to be a mortgage guy. I took a class in C# and started doing development. Then about 3 months ago they made me an Architect.
This is how the term Architect has become entirely meaningless.
|
|
|
|
|
Architect is certainly meaningless in that bank. Maybe he was a forex trader who blew up his account but knew where the bank had buried the bodies, so they had no choice but to move him into a senior software role.
|
|
|
|
|
raddevus wrote: But it is too bad (and probably a part of the entire problem) that there isn't a good measurable way to distinguish between a junior and another role.
There is, it's called experience. Not something you can learn your CV bot 
raddevus wrote: They never attempt to describe why someone is an "architect" versus a junior.
It isn't. They no care, they want the cheapest that gets the job done. So a junior with 20 years of experience in .NET Core.
raddevus wrote: I asked him, "how long have you been using C#?"
Since C# 2.0; before that, I was a Delphi wizard and took out a company by accident.
raddevus wrote: This is how the term Architect has become entirely meaningless.
Even with my experience, if they ask for an architect I first ask about scope to make sure it is doable. If it ain't, I advise to hire someone "better".
Bastard Programmer from Hell
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
|
|
|
|
|
If the failure rate is 70+ percent, and has been most of the time, it's not appalling, it's a standard.
It was only in wine that he laid down no limit for himself, but he did not allow himself to be confused by it.
― Confucian Analects: Rules of Confucius about his food
|
|
|
|
|
If you were to remove the title, someone else would be doing the same thing, just under a different title.
From my own experience, someone keeping a bird's eye view of all the TECHNICAL stuff involved (so no, product managers don't count) can be a huge gain for a complex software project. By "complex" I mean anything from "it actually consists of 3 entirely separate pieces of software, working together through different physical interfaces" to "regulatory requirements are another thing to keep track of, lest you'll be unable to sell that thing".
|
|
|
|
|
After 25 years in the industry, I've finally attained the title software architect. I'm one of two in the organization of 35 developers (single product).
In this organization, it is my job as architect to look at the big picture of software design; what is good for the application five years from now. We are an agile shop and, at least in our flavor of agile, while it has a lot of good going for it, it also encourages tunnel vision. The average developer is focused on delivering today's story without too much attention to how that implementation fits into the overall software product or the future of the software product. That's where a software architect comes in.
I'm guiding the future implementation (the "how", not the "what") to a common set of design goals. I wish every developer had some architecture chops (not all developers are created equal though). But, even if they did, someone has to make the design decisions that effect the entire application. In my designs, I usually always give a sample implementation though. I do some UML if appropriate, but more important is the design documents, presentations, and samples I provide.
|
|
|
|
|
The book sounds pretty dubious, but recently I've been exposed to architects and builders in the context of a real building.
Architects deal with such matters as how space is used, or how to make a space so that it can support various kinds of uses. They're aware of dimensions, of materials, of light, of acoustics, of cost, of functions, of aesthetic, of legal boundaries, of stakeholders, of how people interact with spaces, and all that sort of thing. Ultimately they are great at devising trade-offs and their knowledge is extensive.
Builders however, when told to build a wall right there, are thinking about quantities of materials, where to get them from, of how things will joined, of angles and frames and cladding and hinges, the order of operations, whether the stuff will fit through the door on the way in, and who's going to hold that while another person fastens it.
It's actually quite amazing watching how they are so complementary yet also quite ignorant of the domain of the other specialist. Yet somehow the building gets done.
That co-operation is partly facilitated by a drawing, and partly through them each each talking to the same stakeholders. But wait, there is another specialist too! The draftsman makes a drawing, and he/she is actually very selective in what each drawing is 'about' and 'for' and therefore what it includes or excludes. A set of drawings is typically needed to establish context, indicate dimensions/boundaries/locations, label space usage. These drawings will each focus on different features/aspects of the building.
One of the most marvellous books I ever read that I felt got close to this 'plan' is now out of print (last time I looked): "Problem Frames" by Michael Jackson (no, not that Michael Jackson). It conceives of domains and machines connected by phenomena, and explicitly describes the software task as "configured this machine to ...". So when writing a browser app, it would very specifically call out the user browser (or the mobile device, if warranted) as a machine in which the programmer needed to produce some effect, in order to satisfy some requirement.
Anyhow, if we're going look to architecture/building as a model for getting complex projects done, we need something like this is needed to fulfil the role of the 'plan', and the Problem Frames book is a good start.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
