by Prof Barry Dwolatzky
In my role as Director of the JCSE at Wits University I have, over recent years, been interacting with a large number of South Africans who earn their living as software developers. In some of these interactions I’ve had the opportunity to formally assess their skills and abilities. I’ve been shocked to find that some – a small but significant minority – of our “professional software developers” find it difficult to write working programs – even simple ones. Some of these developers have degrees and diplomas in IT-related disciplines from local Universities and Universities of Technology.
As an academic responsible for educating software developers, and as a person deeply concerned about the successful future of the South African software sector, this situation has troubled me deeply.
What is a “software developer”?
Before I go any further, let’s be sure that we’re on the same page with respect to terminology. I understand the term “software developer” to refer to the individual at the sharp end of the software industry entrusted with the task of putting hands to keyboard to write software. In the past we called these people “computer programmers” – (I’m not sure when and why they were rebranded as “developers”).
We can debate in some detail the set of generic and specific skills that a “software developer” should have (I will come back to this later). There is however one absolutely critical ability that (in my mind) is never negotiable. A software developer must be able to write a program! In other words he or she must be able to convert a specification or requirement into a set of instructions in a “programming language” (of some sort) which can then be run successfully on a “computing device” (of some sort).
Although this may seem obvious, let me expand on this last sentence.
Suppose, as a very simple example, that the “programming language” is C++ and the “computing device” is a PC. The specification might say “Read a set of numbers from a text file. Add them up and write the total to another text file.” Anyone who calls him/herself a “C++ software developer” must – in the very least – be capable of translating the specification to (something like) the following:
The C++ developer must also be able to get this program to run successfully on the target computing device. This will require compiling, loading, testing and (possibly) debugging the above program.
In modern software development, there are many different languages, environments and devices. Developers need many specialized skills and abilities. However, any developer, must be able – in the very least – to do something conceptually equivalent to the task described above. The target language may be very different from C++. The target device may look very different from a PC. The specification may look very different – it will undoubtedly be much more complex. But every software developer needs to be able to successfully translate a specification into a “program” that can run on a “computer”. I’m sure that no one will dispute this.
Other generic and specific skills
Given that a “software developer” can write a working program, what other skills should they have? This is a topic that I’m sure we could debate for a long time. My suggestion would look something like this:
Generic skills and abilities
- Detailed design: Any software developer should be able to develop a number of alternative detailed designs that would satisfy a given requirement. He or she should then be able to examine these alternatives and select the one that best meets the requirement. In many cases the requirement may correspond to a small piece of functionality and the design options may be relatively simple.
- Unit testing: The developer should be able to test the program that (s)he has developed. Test cases may be given as part of the requirement. They may also have to be devised by the developer. In either case the developer will need to run the tests in a systematic way and record the results.
- Documentation: The software developer is responsible for low level documentation. In many cases these are added to the code being written in the form of comments.
- Estimation: If asked the question “How long will it take you to write a program to do X?” can the developer provide an estimate? (S)he should be able to do so with some degree of accuracy. Furthermore, what is this estimate based on? Is it an “educated guess”, or is it based on data collected by the developer from previous tasks?
Specific skills and abilities
- Different software developers need a vast array of specific skills relevant to their specific technical environment. They also need to have some understanding of the domain (eg. financial services, health care, etc.) in which they are working. Many of these skills are acquired “on the job”. They are not taught at universities and colleges.
The problem we face
As I said in my introductory comments I have a concern that some of our local “professional software developers” do not have the “entry level” generic skill of being able to write a program. If this is true then none of the other generic and specific skills really matter.
My concern, however, is based purely on a perception gained in interaction with a number of developers over recent years – it is not based on any research I’ve done. I may be wrong!
My question to you – the readers of this blog is: Do you share my perception? Can all professional South African software developers write a program? If (in your experience) they can’t, what should we be doing about this? Is it a problem? Am I wrong to assume that “software developers” need to know how to program?
I would welcome any comments on this issue.
The Princeton Review says: Software developers coordinate the production of software products, from choosing content providers, assembling graphics creators, and working with programmers, through the actual assembling, pressing and distribution of the final product. (http://www.princetonreview.com/Careers.aspx?cid=145) This suggests the developer is more like the property developer, who does not lay the bricks.
Wikipedia is less certain of the role (note the use of the word “may”): A software developer is a person concerned with facets of the software development process. They can be involved in aspects wider than design and coding, a somewhat broader scope of computer programming or a specialty of project managing including some aspects of software product management. This person may contribute to the overview of the project on the application level rather than component level or individual programming tasks. Software developers are often still guided by lead programmers but also encompasses the class of freelance software developers. (http://en.wikipedia.org/wiki/Software_developer)
My view? Whether the developer will write any or all of the programs, or not, the developer should know HOW to write them. This knowledge will make them a better developer. I see the developer role as the harnessing of appropriate resources to create the desired outcome – within the constraints of cost, time and quality.
I think universities/technikons should be held responsible for this (or lack of). A graduate should not be allowed to graduate if they cannot complete a (software) project. That is how we got our qualificaions, I do not know why it stopped.
“Detailed design: Any software developer should be able to develop a number of alternative detailed designs that would satisfy a given requirement”
Well, when developing object-oriented software one tends to alter the model or even replace the model. After a couple of models (apparently statistics put the number at around 3 — strokes quite well with my own experience for something slightly more complex) the resulting model should be able to do the job. However, it would not be possible to come up with these models up-front and then choose one. But I figure you are not suggesting that.
“Unit testing: The developer should be able to test the program that (s)he has developed.”
If there is a progam the developer surely can test it. However, using test-driven development is a skill that is desperately required in SA (well, I guess everywhere). Automated unit and integration testing (along with some decent CI) would go a long way to stabilizing software.
As for low-level documentation I fall in the school of thought that code should document itself. There are cases where comments *may* be required but these should rather be the exeption. Rather than comment something like:
DateTime.Now.AddDays(-7); // get this day last week
one could encapsulate the functionality in a method called ‘ThisDayLastWeek()’. Martin Fowler’s refactoring book is quite a good example.
I also subscribe to the Agile ideas so documentation should be kept to a minimum. I am not religeous about this and when required to produce documentation by a client I do so. But it does take time and all-too-often some clients think that documentation is an auxilliary function that “just happens”. But sadly, it actually *does* take time.
As for estimation: it can rarely be done with any amount of accuracy in the real world. A thumb-suck is fine and we’ll throw in some burndown charts and get into stride with a team “velocity” but estimating duration when it comes to software is damn near impossible. Given a good couple of years of software development one may have a gut feel when working with known technology but when a new technique or technology is to be used we are back to thumb-suck.
Lastly I also believe that the code is the design.
Just my ZAR 0.02
But to answer your question
“My question to you – the readers of this blog is: Do you share my perception?”
Yes. I for one *do* share you perception. However, it is probably worse than most folks think. The thing is that when we get to software of any meaningful size / interaction there is a lot more to it than simply “writing a program”.
But I guess your concern is not really at that level. So if that entry / junior level is as bad as it looks… well… it doesn’t get any better up the ranks.
Many thanks to Barry for his very relevant questions and remarks. The thread of replies is growing quickly, which indicates that he must have hit a sensitive nerve.
Because I am not an industrialist, I am not in the position to answer Barry’s question which was clearly addressed at the industrialists (not at the academics such as me).
However I would like to remark that any industrialist/s, who has/have more than only a little bit of anecdotal evidence in this context, could write and send a paper (category: industry-oriented papers and case studies) about this relevant topic to the upcoming SAICSIT’2011 symposium.
Paper submission deadline for SAICSIT’2011 is the 20th of May. For the whole call for papers, including all instructions, please see http://dl.cs.uct.ac.za/conferences/saicsit2011/cfp
Moreover please note that SAICSIT’2011 is also calling for panel discussion proposals. The various topics from Barry’s Blog would all be interesting for a lively panel discussion in the context of SAICSIT’2011.
Kind regards from Pretoria!
Hi Barry
Very relevant article, and something I experience on a regular basis. I cannot really comment (in depth) on aspects of software such as design, testing, documentation, but I do know that our programming skills in general appear to be quite poor (based on my anecdotal evidence). I interview many programmers, and it is fairly clear that the “basics” is not all that well understood (basics of the language – in our case, Java, aspects such as OO, error management, etc).
I have also found developers (and some with a few years of experience) who struggle with simple algorithms. My sense is that most software developers get very little guidance in their early working years – most organisations appear to either ignore most of the people development aspects, or they are asuming that the tertiary institutions are doing ALL that is required – I am not really sure where the problem lies.
My sense is also that people are allowed into tertiary level education for degrees/diplomas in software-related fields without the necessary acumen. Some of these graduates are also put through programmer learnerships and still end up in IT fields (and in some cases, non-IT fields as well) not related to programming.
There appears to be no real structure or programme in place to get the average programmer to the next step (i.e. that of righting robust commercially viable software). Perhaps we need a create an “articles” for a programmer before he can be called a professional softwate developer.
As a country, I also think we do not write enough custom software – it appears that we are doing a lot more integration related software development – I think this is also a contributing factor.
I am currently doing quite a bit of mentoring of programmers, and I am hoping, in some small, this will make some difference
Regards
A short reply to Mvelo Walaza (RE: “I think universities/technikons should be held responsible for this”).
Let us please not enter into a political style of blame-game in which always THE OTHERS are responsible for any observable misery.
Everybody carries on his shoulders some share of the whole responsibility.
* Pre-university Schools: Are they preparing their pupils well for the universities?
* Universities: Are they doing their best in the education of their students? Are the universities sufficiently producing not only commercially employable engineers, but also good school teachers for the education of the next generations of pupils at pre-university schools?
* Industrialists: Are they doing their best in enabling further in-house staff-development, and are they donating some of their commercial profits in the form of grants and bursaries for the improvement of education in schools and universities? Or are they donating nothing, in order to only maximise their immediate commercial profits?
Last but not least, also each and every individual has the professional-ethical obligation to engage in continuous self-improvement. The salary of any IT-employee is sufficiently high for buying a new book every now and then, and everybody who can find the time to watch a football match on TV can also find the time to read a few book-pages in the evening before going to bed.
This principle of continuous individual self-improvement is also expressed in the IEEE Software Engineering Code of Ethics, Principle 8 (“Self”): “Software engineers shall participate in lifelong learning… Further their knowledge… Improve their ability…” (etc.)
As Chris says companies have to go the apprenticeship route. First off you obviously need to get buy-in from both the senior and junior developers.
The junior watches (pair programming), listens, and is assigned tasks that are reviewed. The senior is still ultimately responsible for the quality of the work and project commitments.
I would be prepared to mentor but only after the junior has successfully completed a rigorous aptitude test to prove they have problem solving skills. I have seen many graduates from our universities that are unable to create an algorithm for the most simplest of problems, that I could do in high school.
A reply to Mvelo Walaza – From my experience, the best way to learn how to program is by spending lots and lots of hours in front of your computer. Figuring things out and making programs work.
Universities can only teach one basics but it is up to an individual to learn more.
Agree with most of the comments………..The fundamental is aptitude and logical thinking ability of the individuals.
Thereafter rigorous training with periodic qualification tests to categorise skill level….’platinum’, ‘gold’,…..etc.
Have not seen an old practice of code walkthru being followed religiousely now-a-days…..someone can argue about the more and more integration work, package implementation , customisation work in the field rather than custom bespoke development……However, the basic discipline inculcation lies with seniors as mentors so that it becomes a work culture where the young talents are nurtured after their college days…….
It applies to writing program specs, coding standards, test specs etc…..
In sumamry, the workplace culture should create a learning environment with an accoutability of instilling software engineering discipline………..
By the way – looking again at the yellow highlighted example program of above: There is no need to keep the OUT file “open” while the numbers from the IN file are being accumulated. If – for whatever reason – the system would “crash” while the loop iteration is being executed, then the status of the OUT file would be unneccessarily “dangling”. From a system’s perspective it would be considerably safer to “close” the IN file immediately after the read iteration, and only thereafter “open” the OUT file, very briefly, only for the purpose of writing the total result. Ditto, there is no need for keeping the IN file “open” whilst the total output is written. With such a nifty example I would have explained to my students the subtle difference between “functional” and “non-functional” requirements! From a purely functional perspective it is irrelevant that both files are open for such a long time (in this example), however from a non-functional (quality) perspective the unneccessary simultaneous open-ness of both files, for such a long time, can be a matter of concern.
This also relates to the comment of above, by Lalatendu Mishra, about the value of “walk-through” exercises.
@Stefan / All,
Well, that is what is known as splitting hairs
Although, you *do* have valid reasons. But when one gets down to it there is quite a bit missing from the example — because it is just that: an example.
- There is no error handling.
- There is no validation.
- There is no business meaning.
What happens when, on line 23, there is not a number but rather the letter ‘A’? What do we do?
In what state does that leave the ‘system’. Well, there is no system *here* but let’s play along. Is this totalling exercise part of a larger process?
Let’s say someone decides we need to send an e-mail to Bob the supervisor when something goes wrong. What happens when Bob is on vacation? How about sending to more than just Bob? Should we store a list somehwere? In a database, in our application configuration file, or in an XML file?
What happens when the “powers-that-be” decide that not only do we need to e-mail, we also need to let our CRM system know about this failure, as well as letting the reporting system know when it is successful. Well, this is job security. We will simply alter the program. This leads to tighter coupling between the systems. Maybe we talk to the other system’s database or through a web service or maybe WCF or some REST interface. Still a *lot* of coupling.
A nice idea may be to use a service bus. We will publish the business event and any system (or bridge) can subscribe.
Long story short: *nothing* is ever this simple
Regards,
Eben
@Lalatendu,
Again, not simple.
The problem with awarding some certfication level is that IT is *very* wide. I am a C# developer with 16+ years experience. Been on .NET since 2003. How would you rate me?
I don’t know a thing about BizTalk (OK, I know *some* things). I have never developed a system using Workflow Foundation. I don’t know anything about Sharepoint (apart from it being abused beyond belief). I have looked at WPF but it will probably die. Silverlight was still-born and having asked (in an interview) why a prospective employer would go for Silverlight for their LOB system as opposed to HTML5 it actually cost me the job. And today SL has a very iffy future. You go to a client and they use FileNet or Staffware or Platterpus or Gobsmack. You have to make do.
Each technology has a level. New technology = start from scratch.
I have been asked ludicrous interview questions that in no way test my design ability.
I suggest folks interested in this kind a thing have a look at the ‘Dreyfus Model of skill acquisition’ — very interesting.
Regards,
Eben
@Eben,
RE: “How would you rate me?”
As far as I know, the language C# was developed and published in the years 1999/2000. If this is correct, then it is not possible to be “a C# developer with 16+ years experience”, because we are currently only in the year 2011.
Presumably you wanted to express that you are a software developer with 16+ years of experience, and that good knowledge of the language C# is also included in this long-term experience of yours.
Once again I am indulging into this kind of scholastic “hair splitting”, because your example is very nice in showing us the lack of precision in natural language. Thus, if software specifications are only presented in natural language, there is almost a 100 percent guarantee that the programmers will mis-understand it, and then they are going to program something which they were not meant to do.
Thus, coming back to the education theme of this discussion thread: Indeed I am continuously trying to make my students aware of the lack of precision in natural language, and to this end I am also confronting them systematically with ambiguous natural language sentences similar to the example which you have provided.
However I would disagree with your opinion that Barry’s given example program would not have any “business meaning”. Building a sum of numbers, and storing the result number somewhere, is indeed a very useful operation in many contexts of application.
@Stefan,
“I am continuously trying to make my students aware of the lack of precision in natural language”
I buy that 100% since you totally misunderstood what I was trying to get at. I actually did state that I have been on .NET since 2003. But, in case you were not aware of it, C# is a .NET language. Although I have only been coding C# since 2007. So I did not state that I have been developing C# for 16 years but I would generally think that one can deduce this given the relevant facts. So one needs to understand what the natural language is trying to convey.
And the example is indeed without any business meaning. Building a sum of numbers is *not* a business meaning.
Determing the balance of a customer account is business meaning.
Hope that helps.
Oh, that depends on our notion of “business”
Commercial business is not the only business. If the business is mathematics, then the addition of numbers is meaningful business, from the mathematician’s perspective 
Moreover I think that programming is scientifically interesting in itself, and does thus it does not need to justify its own existence utilitaristically by having to be externally meaningful to other “business”. An interesting program is an interesting program, from a computer science point of view. Whether or not the program also has some additional meaning for any other business – that is a completely separate question, which is however outside the domain of pure computer science.
With a little luck most software is useful
I was trying to demonstrate that the example code is just that: an example. Now if folks calling themselves a programmer cannot read the example or cannot code this in their language of choice we have a problem. I just took it a step further since one will be lucky if you come across something as simple as the example in an enterprise application.
Although computer science is interesting, and probably more so after a good couple of years of software development, it is probably quite a bit removed from what happens in a typical business application (commercial domain — to be accurate).
IT is just so huge that it is *very* difficult to accurately define everything. This is why we will have a hard time getting to where other enginering disciplines are. That is if you are one of the folks defining software development as an enginering discipline.
That is why it is near impossible to get anywhere near consencus on most ideas, issues, designs, or solutions. It is actually quite disheartening