One of the reasons is that there are many different problems, needs that cannot be met by a single programming model. But this involves a certain degree of fallacy, I’m going to go further down. The official reason is this, the others are secondary. For this to be exercised properly the language needs to be of a paradigm different.
Save something very punctual languages are not improved to suit hardware evolution, at least not for technical reasons, since for political reasons the hardware vendor may have the adoption or creation of new language. Also there is no such thing because of operating system.
At some point we can say that it is because there are nails and screws and so it needs different tools, but when the handle is plastic or wood, or metal, or is curved or has other characteristics is because there are people who like it differently, not because it is necessary.
A secondary point of this is that languages always have flaws and a new one comes to give a solution to these failures. Many of these flaws are subjective, which always allows someone to find a reason to do something different. There is no way to make a perfect language, a positive characteristic tends to prevent another or creates a negative.
Is one language difficult to manage memory? It creates another that makes it simpler. Does a language require writing too much code and expressing what it really is? Does the language not allow you to write very large codes with ease? A language is legal but in exactly constructed form is inefficient? All this can be solved using the basis of what already exists, ma changing some details that solve this problem.
Another reason is that people want to have a certain dominance of the market or mark their position in history, so they find something that can be done a little differently even if it’s not that different. This can happen because people tend to like things because of little details.
Why are there so many brands and varieties of so many products? For the reasons I mentioned above, then programming language is no different. Why do you need a white car and a blue one? It doesn’t change anything in people’s lives.
Of course, a new language solves a problem found in another language, even if this problem is not so relevant, but the biggest issue is that people want to do different things. It would make sense to have a few very different languages, the rest is just a space fight. It is even a partial fallacy this question that each one meets a different need. Of course this is partly true, but part is just to suit a different taste. Except in some scenarios it makes little difference to use one thing or another.
So the real reason is that people like different things and there are people who want to supply that.
Why are there several types of computers? Or cars? Many of them do not actually exist anymore because they became obsolete, they were good at the time they were created, but today no more. With language it is the same, with the difference that it does not wear out with time and can still be used.
Again, whoever created something in the past tries to keep their product relevant and makes improvements. This only works because have those who like.
In essence all programming languages can do everything necessary and some are slightly better at some specific point. Often you see more difference in the implementation of one language than from one language to another.
Languages are forms of expression so each has a preferred way of doing it. In some cases this way may be easier for certain scenarios. It is not so different from natural languages. Even some serve better than others. Some serve better for technical subjects others serve better for artistic scenarios.
Have you ever noticed a great strength of people trying to evangelize about the use of a certain language? People know that greater adoption leads to greater adoption and this is necessary. Most adoptions are made by marketing, even spontaneous, and not by the actual technical characteristic. Of course, it is rare to adopt very out of parameter, but it is also rare to have something that is very complicated to do in different languages.
Currently there are thousands of programming languages and each has the right to create their own, what will determine whether it will be known is how much you can disclose it and get hold of other people. And thousands are used, only varies the amount of use.
Many languages are not created to be viable products but rather experiments or evolutions of any existing.
I often saw people adopt a language not because it was different but because it had a library that people wanted to use that way. Even some think that a change in the library creates another language when that is not true.
The fact is that there are literally infinite combinations possible to make in one language. Each sees one combination as better than another.
It makes it even easier to evaluate language against language about it. For example, there’s a reason people use C or C++ and it’s not performance as many people think, only naive people think it is. I’m going to try to put some motivations that we see around to give an overview of how choices are made. I’m of course making simplifications.
C was created to be a portable Assembly. And be something easier to express than ALGOL which is a fantastic language but very boring to use.
C++ was initially created to add C classes, to give better abstractions without paying anything for it.
Java was created to be a better C++ (never got anywhere near that, it’s worse than C++ in almost everything that doesn’t involve robustness).
C# was created for Microsoft to have an alternative to Java since it was forbidden to evolve Java. And also to be what Java said it would be, ie a C++ better, so it got in the middle between C++ and Java.
The big difference from these languages to C++ is memory management.
VB.NET is the VB for .NET. VB is the BASIC most suitable for Windows and that Microsoft could control the evolution. It has an immeasurable amount of BASIC dialects. This language emerged to be a simple alternative to what existed around as FORTRAN or COBOL.
F# exists to have a working footprint on . NET, based initially on Ocaml.
Pascal ALGOL should be more pragmatic.
Delphi would be Pascal with classes, an attempt by Borland to keep its market around Pascal in the face of the novelties that emerged.
Objective C was created out of some dissatisfaction with how C++ handles object orientation. Swift was created because Objective C had too many problems and still held the legacy of C.
R was created to facilitate statistics. It is one of the few languages that imposes a real difference that makes sense to exist beyond taste.
Rust was created because C++ has many flaws and Java and similar have tracing Garbage Collector.
Go was created to be C with GC and correct some errors of this language, in addition to allowing better abstractions (but not so much in some cases), and also to compile fast. Just like in other languages, the market goes elsewhere and the goals change. Not always can the creator keep the language the way she thought, people can adopt it, but have other ideas.
D was created because C++ was too complex. It didn’t work out so well and they are reformulating the objectives.
Dart was created because JS was very bad.
Scala was born to have a more functional strand for Java
Kotlin emerged to have an alternative without the problems of Java running on JVM (had other goals initially, but were abandoned).
Eifell was created to demonstrate that it is possible to make multiple inheritance and how is the use of contracts.
Haskell emerged to teach Haskell to people :P :D
They say that V was created to raise money from unsuspecting people (I’m not the one who said that :) ).
Javascript has emerged to allow customizing web pages with behaviors, and so it should be a simple and easy-to-use language, not to make applications. Most of the choices made for her were for lack of choice, or at least lack of time to choose to do better things.
Typescript was created to put types and more organization into JS, thus allowing applications.
PHP was created to facilitate the work of creating web pages where the C that was used by the creator was very complicated.
Python, Ruby, Perl and others have appeared to be something similar, or at least to be able to do scripts simpler than C, Pascal or other language of the type, but more powerful than awk, bash and others shells. Ruby wanted to put OOP on scripts, beautiful idea.
Python for example tried to give an air of functional language but with the imperative way that everyone is accustomed, which is a recipe for success, because functional languages are simple and beautiful, but bad for certain usage patterns. Hence the indentation to define block.
Groovy exists to be a dynamic alternative to Java.
Moon was created because they needed a Brazilian technology to make scripts simple and with performance embedded in other applications.
Boo should be a better Python running on. NET, but died from lack of community and commitment from the creator, perhaps because C# improved much of the things that Boo brought.
Elixir should be dynamically typed Erlang among other "improvements".
dBase was created to support a simple database. Clipper was initially created as a dBase compiler, but then I need to evolve to be able to create complete applications. Harbour was created to meet those who became orphaned of Clipper and ended up having a natural evolution to meet new demands. Foxpro should be the Clipper of another company to earn some of the rivers of money that Nantucket was earning. It has other dialects created each for the interest of a company.
These latest languages usually give up efficiency to give more ease. Some less than others, an example is Lua that can be easy and simple without being too inefficient.
There are some languages that nobody knows because it was created because it only makes things worse :) In general it’s just the creator’s taste.
I find it curious how some languages were created with a clear goal and was changing, today is something else and nobody notices. Many people defended language as it was and today defends the other way going against their initial argument. People root for teams, political parties and companies and technologies, all in a religious way but with dogmatism biased to the discourse of the moment. This gives space to emerge new languages, people are waiting for a speech that makes more sense to her.
Some languages are true Frankensteins.
There are programming languages more or less indicated for each of the multiple classes of problems to be solved. Now you’re being very modest, the last time I researched there were over 2,500 different programming languages
– anonimo
I read over the answers, but did not see comment, so I will comment here, there are languages simply created by some personal "whim" (php I see as an example of this) or entertainment (usually "esoteric" languages). In the case of PHP, Rasmus created it for his personal use and it was really different from what it is today, in fact his idea about languages was sometimes a little mistaken, but I’m not going to criticize, the focus is that the language was adopted by a "community" and they evolved it, that is, no longer what it was [...]
– Guilherme Nascimento
[...] then his initial reason was not to solve something that another language could not, was initially to be something personal that he might well have done with another interpreter, as was common at the time, perhaps Perl (at the time usually used CGI (port/protocol))... Other possible (over other languages) motives are probably the "commercial/marketing", ie a technology company that develops its own language should be something "amazing" in the eyes of the "world" (at least so people expect)
– Guilherme Nascimento
To celebrate the question, I was curious because I saw this video yesterday https://www.youtube.com/watch?v=Og847HVwRSI
– MarceloBoni