1. Greg Pfister, IBM, <pfister at ibm.com> initiated the discussion by posting the following:
Does anybody have a pointer to a history of parallel and/or concurrent languages? Unfortunately, this is a time-critical request; I need to give a talk on it in a week (company internal, but I'll of course share what I'm helped on with the community). There are a number of computer histories I can find; for example, Gregory V. Wilson's was particularly useful, but it mostly ignores languages. There are sites that do list many languages -- "Nan's Parallel Computing Page", Jonathan Hardwick's list of research groups, and the Internet Parallel Computing Archive, for example. But they tend to simply list efforts in alphabetical order, and seldom provide dates and origination information. Any pointers will be appreciated. Thanks. Greg Pfister IBM Distinguished Engineer, Member IBM Academy of Technology.
After receiving lots of responses from TCSC members to his posting, Greg writes:
Wow, thanks, all, for the pointers [to history of programming languages]. And the blasts from the past; I guess that's inevitable with history. I can't believe I'd forgotten about Almasi & Gottlieb; I have a copy right here. And Sammet -- not here (home), but in my office, I believe. Good stuff to mine, and I'll be doing so. In the meantime, one good reference I found and a further question.
The reference: "The Development of Data-Parallel Programming", by David Carpenter, http://www.hpjava.org/talks
The question:
With just an hour or so of web surfing, I amassed a list of 80-100 different parallel language efforts, and those are mostly current, active, efforts. (I wondered for a while why I wasn't getting many web references earlier than about 1993. Duh.) There are probably at least another 100 pre-web.
So why, as a first approximation, are none used? Sure, some are used somewhat, in some cases. But go to IPDPS or the like, and all you hear about is (a) MPI - mostly; (b) OpenMP - much less so. Not even much on auto-parallelization, recently.
My tentative theory: It boils down to $$$$, via portability and longevity.
A good compiler, parallel language or not, is expensive to develop. The customers aren't satisfied with a mediocre one.
Similarly, important application codes are expensive to develop, and are expected to last a long time, well past the hardware fad or acquisition of the moment. (Come to think of it, compilers themselves try to last a long time, too.)
So, nobody puts the application code investment into anything that's not extremely likely to be portable over machines and over time. That boils down to a very standard language (Fortran, C) with a subroutine package (MPI, OpenMP). New languages may be nice, but ensuring they're on many machines over time is at least not simple and at worst very expensive.
There are undoubtedly other reasons, like education and familiarity. But I think they pale compared with the economics of portability.
Any comments on this?
2. Richard Enbody of MSU <enbody at cse.msu.edu> replied:
How about Amdahl's Law?
I have found that wringing sufficient parallelism out of existing algorithms which are not a so-called "embarrassingly parallel" problem involves so much application-domain knowledge (for algorithm rewriting) and fine tuning that a parallel language probably wouldn't deliver. If starting from scratch was an option (and hasn't been for my purposes), a parallel language might be an option, at least as a starting point. (If for some bizarre reason you wanted to read of an experience, see Enbody, R.J., Horoi, M. "Using Amdahl's Law as a Metric to Drive Code Parallelization: Two Case Studies," International Journal of High Performance Computing Applications, Vol. 15, No 1, Spring 2001, pp. 75 - 80.)
3. Larry Rudolph of MIT <rudolph at csail.mit.edu> pointed to a Catch-22:
A small comment on your "success of language" observation. The success of a programming language is measured in terms of how much it is used. The more programs written in a language, the more successful it is.
It is thus very hard to "sell" a programming language, since people will only buy one, if it is successful and it will not be successful unless people use it, and people will not initially use it if it costs money. So, this catch-22 leads to the development of open-source efforts, and these are hard to develop good compilers.
I have expanded my research to pervasive computing but still have some ties to parallelism. Whenever I look, people keep saying the problem with parallelisms is the lack of good languages. I always found this a silly comment. I guess the more appropriate comment is "the lack of widely used parallel programming languages"
One exception is the parallel mathlab language developed by Alan Edelman (I told him to write to you).
4. Alan Edelman of MIT <mit.edelman at gmail.com> followed up his colleague’s request with:
Can you add me to the mailing list? My colleague Larry Rudolph has been forwarding the emails.
I have two reasons for lack of new languages.
1. People don't want to program in a new language. They are psychollogically resistant. This can be overcome if everyone around them tells them this new language is great, but then it better be great and available.
2. I did an experiment some years ago with parallel software. I asked students to download some parallel software and see if it can work. (It often didn't.) Then download another piece of software and have them work together. (It almost never did.)
I think you can draw your own conclusions.
Here's a survey of almost 30 parallel matlab's:
http://www.interactivesupercomp
Here is gridmathematica
http://www.wolfram.com/products
Here is hpc-grid maple
http://www.maplesoft.com
These high level languages are the best chance of making the difference (with star-p on top of the list in my humble opinion) :-)
5. Allan Gottlieb of NYU <gottlieb@nyu.edu> brought up issues of familiarity and quality:
As you mentioned, the issue of familiarity is certainly important. At the least users want the name familiar ("I don't know what the dominant language for scientific programming will be in 10 years, but I do know it will be called fortran").
Also there is always the question of high quality compilers producing good quality object code. Backus was certainly right about that, although his efforts into parallel languages were not nearly so widely adopted. This ties in to another point of yours, the requirement for portability. However, performance portability has not been so successful. When one of the labs gets a new supercomputer, the old codes can be made to run fairly easily; but to fully utilize the new machine considerable effort is typically required.
6. Marc Snir of UIUC <snir at uiuc.edu> pointed to implementation iterations and architecture:
I agree with the $$$$ reason, and the long persistence of software: people have been burnt by platform specific languages (CM-Fortran, for example).
One more reason: it takes several iterations to get a language design right, and several iterations to get the compiler technology right; there is no market to support this iterative process.
Minor correction: OpenMP is supported by a compiler, not a library (of course, it has a run-time library).
Observation: Any major new language paradigm in parallel processing came from the need to exploit a new architecture that offered significant performance benefits: vector, shared memory parallelism, message passing. In each case, standards followed many years of proprietary solutions. I.e., when there is a clear performance drive, then there is a market to support the iterative process of experimentation, and people are willing to break compatibility to get to a new level of performance.
Even in the sequential world new languages are very hard to introduce, despite the much larger investments: C++ came in because it was compatible with C, and supported OOP. Java came in because of the internet and the need for applets.
7. Rajkumar Buyya of U. of Melbourne <raj at csse.unimelb.edu.au> points to Redmond:
Looks like with Microsoft/Bill Gates entry into HPC/Cluster Computing field, all this is going to change. Look at the press coverage on Bill Gates keynote talk at Supercomputing 2005 conference.
8. Robert Bernenky <bernecky at rogers.com> comments about tools of thought and financiers:
Well, since you asked...
Coming from a place where our philosophy of design was to create "tools of thought" (Iverson), I believe that we should NOT mix algorithms with low-level inter-processor communication code. That's why, for example, I introduced the Rank Conjunction into SHARP APL: it offers a concise, clear, and formal way to spread a computation into independent pieces. It does NOT, of course, provide any mechanisms for inter-processor communication. It's basically a FOR-all that applies to arrays of any rank, rather than giving you scalars-whether-you-like-it-or
Open source can't hurt, of course.
The real money is going to come from people like the merchant bankers who want to be able to convince themselves that their arbitrage and FX programs do what they think they're doing. That's why those people go for array languages, in spite of their boutique reputations.
9. Niraj Srivastava of HP <Niraj.Srivastava at hp.com> agreed with Greg:
Regarding the question:
You are right on the money, it's all about portability and longevity. Application developers instinctually live by the following two principals:
a. mean run time of the application is longer than MTBF of the machine.
b. mean age of the application is longer than the life of the machine.
These rules lead to an emergent behavior that seems anti-intuitive; however, it does maximize portability and longevity. If one wants to be able to quickly move applications from one architecture to another, you can't rely on fancy compilers and tools. They are always 6 six month behind the first availibity of the machine and then they are only 99.9% compatible with the previous architecture. The 0.1% is the killer. WE in the HPC industry have not done much to make the end users life any better.
Can open source help? Not clear how one defines a community for some thing like this.
10. Horst Simon of LBL <hdsimon@lbl.gov> shared some lessons from an NRC report:
I think you have one part of the situation analyzed correctly. Thousands of applications users who have settled on MPI, because it has proven to be constant in world of rapidly changing large scale platforms and system. Any new language today is simply too much risk, when progress in the scientific application is at stake.
But there is more to it. Actually the NRC report on the "The Future of Supercomputing" (see http://www.nap.edu/catalog
The "economics of portability" is only one link in a loop, that has to be changed simultaneously in order to make progress. Today we have more than ten years of legacy applications development in MPI, and inexpensive, but mostly very effective clusters that match the MPI model perfectly. This is an eco-system in balance that will be very difficult to change.
11. Duncan Buell of the U. of South Carolina <BUELL@engr.sc.edu> added experience to Horst’s comments:
At the risk of beating a dead horse, let me echo Horst's comments. In the 1980s, there were dozens of parallel computer vendors. Each vendor had a different set of pragmas and special methods for getting high performance from the particular architecture.
And nothing ported. The result was that nearly all benchmarks were incommensurable with each other. Everything was an ad hoc "if you do X with this machine, you can get performance Y". One could do some analysis of the underlying cause for performance (good or bad), but none of that was very relevant for very long because what worked for an Alliant wouldn't work for a Sequent; what worked for an nCube probably wouldn't work for a Hypercube.
And then version 1 of the hardware became version 2, with a different set of methods for getting performance. Some vendors even did 180s in their religious dogma--sometimes more than once.
There were a lot of data points. There was precious little probing of why machine X with feature Y didn't quite work, but machine Z with a slightly different kind of Y prime actually did work.
And so the vendors died one after another. If I had been middle management with operational problems to solve, I would not have felt it worth the investment to keep rewriting code.
For good or for ill, MPI has been a reasonably standard approach to parallel computing that would port to every ethernet-linked group of toasters on the planet. The problem is that the low cost of commodity clusters and the high portability of apps in MPI have discouraged the continued development of anything but a cluster, but in fact there are a few very important applications that don't work at all well on such clusters.
12. Timothy Mattson of Intel <timothy.g.mattson@intel.com> pointed to a completely different kind of reason:
There is another but very important reason for why so many good parallel languages have failed. It's the phenomena of "choice overload". This is a well documented effect in which consumers are chased away by too many options.
The most famous experiment showing choice “overload was conducted at the Dragger's grocery store in the bay area. It was published in the paper
Iyengar, Sheena S., & Lepper, Mark (2000). “When choice is demotivating: Can one desire too much of a good thing?” Journal of Personality and Social Psychology, 76, 995-1006.
They had two jam displays ... one with 24 jars and the other with 6 jars. If a person sampled the jam, they got a coupon to buy the jam at a discount.
They found that the 24 jam display attracted many more consumers to try, but only 3 percent purchased any jam. With the 6 jar display, however, a whopping 30 percent purchased jam. Since that famous study, this team has confirmed this effect with 401K plans ... so it's not just with frivolous stuff such as jam.
I believe that the proliferation of programming languages in the early to mid 90's seriously hurt us. We were trying to attract programmers (i.e. consumers of our parallel computing technology) but we scared them off with dozens if not hundreds of different programming notations.
Choice overload damaged the field of parallel programming.
So I second Horst's statements. But I think it goes deeper. And I think as we consider what to do in order to attract a larger pool of parallel programmers, we should be very careful not to proliferate countless languages in the hope that one of them sticks. Research many different languages, but when it comes time to roll out a technology for the larger application programmer community, "less is more".
13. David C. DiNucci of elepar <dave at elepar.com> brought up performance and funding:
The definition of "portability" you use here apparently relates to getting something to run on lots of different machines without too much trouble, though perhaps at a fraction of its potential speed unless specifically hand-tuned for each or if based on lots of assumptions about absence of faults, sharing, heterogeneity, etc. Using that definition, the degree of portability is (almost by definition) directly proportional to the number of machines that the language or package is implemented on. If such a package is only a standardization of accepted existing practice (as was dictated, for example, by the MPI Forum guidelines), implementation of that package on a new machine will encounter few roadblocks or risk, and applications can then follow, also with little risk. "Standardize it because people want to use it" becomes a self-fulfilling prophecy.
I would argue that "portable" should imply "portably performant". MPI isn't portably performant because, for example, it dictates that data will be copied between processes even in cases where that is not a wise performance decision (e.g. where data can be shared in situ), and similarly, it does not support variable granularity (such as the effective merging of smaller processes into bigger ones where the number of processors is relatively small). Plus, it fails to address issues like sharing, faults, heterogeneity, and irregular apps.
By sufficently lowering performance expectations and sufficiently restricting the machine architectures and application classes under consideration, the "performant" modifier can be omitted, so any standardized interface can be called "portable". It works the other way, too. By defining MPI as "portable", we are severely restricting the applications, platforms, and performance that we might otherwise be striving to support.
I tend to believe now that the funders themselves aren't convinced there's any value to work being done in this area, probably for the reasons you give above. I thought I had already seen every excuse they could dream up to deny or withdraw support for my work in non-MPI directions, including:
* "Focus on either software engineering or parallel computing, not both"...as though software complexity and parallel computing are not inextricably linked.
* "You're trying to address too many problems at once"...even though all of those problems need to be solved to produce viable methods.
* "You're group is too small"...which sounds like a reason to provide funding rather than refusing it.
* "Management doesn't understand it". I would be surprised if management's limited experience in parallel programming DID prepare them to evaluate novel parallel programming solutions, pre-prototype.
* "This is phase I (factfinding) funding, but you already seem to have preconceived notions regarding the form of the solution". Yes, "preconceived" after 20 years of research.
* Similarly, "the ideas were developed too long ago, they must be stale by now". Yes, stale like physics and calculus.
* Similarly, "you invented this long ago, but I've never heard of it." Perhaps because I got this excuse the first time I tried to get funding and every time since.
* "The experts at [famous lab or university cs dept] are working on that". Yes, and they've been working on it for years with $millions in funding, maybe it's time to give someone else a try.
* "We trust that MPI-2 will handle that" (before MPI-2 had even been finalized). Punch line: MPI-2 didn't handle that.
* More generally, "We believe that [HPF/OpenMP/UML/J2EE/BSP/Object Process Method] is already solving all of these problems, or soon will". So anyone still complaining about lacking parallel programming support is just uninformed?
* "Don't bother suggesting a new methodology without also describing tools to convert dusty decks". (Yes, I heard this long before Horst's recent message.) I've never understood this. Why would it be considered a poor investment to validate a new representation sans conversion tools, but a good investment to create conversion tools before the target representation has been validated?
Upon seeing the responses to your query, I assume those excuses will now be complemented with "only MPI-performant machines are being built these days anyway" or "people will just get confused if they see another parallel programming approach".
In this light, the only way I have found to get any work done on promising novel portably-peformant approaches (e.g. Scalable Planning Language, ScalPL, formerly Software Cabling) has been to extract myself from "work" environments that would push me away from my chosen directions, to completely give up on getting funded, and to just work on my own directions while bartering to keep a roof over my head. So far, based on the lack of progress I've seen elsewhere, I think I have made the right decision. Meanwhile, other people who get Phase II SBIR awards are telling me "well, it's not really funding, it's just money to look for more money".
> Could open source potentially change this situation?
It is extremely unlikely. If it could, it would likely have shown some signs by now. Open source is best at developing cheap alternatives to proven designs and implementations, and this situation requires new work.
FWIW, a form of Cooperative Data Sharing has been open source for years now (see http://cds-bcr.sourceforge.net), and as far as I can tell, it's hardly been noticed, in spite of some strong advantages over MPI (e.g. regarding portable performance). At the very least, its interface should serve to provide good concepts for an MPI follow-on (as long as it doesn't need to be backwards compatible with MPI). But it sounds here as though people see no reason to formulate an MPI follow-on.
Bottom line: I believe the parallel computing community in the US will likely move beyond its self-imposed MPI-constrained rut only when customers demand it (e.g. in terms of superior games or other programs which run on networks or massively-multicore chips), or some "rogue nation" scares us into it by beating us to it.
14. Srinivas Aluru of Iowa State <aluru at iastate.edu> connected performance to parallelism:
One important reason why most languages failed to take off is because of their lack of attention to performance. Whatever CS folks think, the only reason parallel computers get used in practice, given their relatively high cost, is to work on large-scale problems that cannot be solved serially because they are memory-intensive or compute-intensive.
When someone uses a PC to run a job that takes one tenth of a second using an efficient but not so elegant language, they would not mind using an elegant language and doing the same computation in one second if that alternative is presented.
15. Frederica Darema of NSF <fdarema@nsf.gov> pointed to research programs in related areas:
You are right there are lots of stuff going on but some the same - OpenMP has quite a bit of the EPEX flavor - you will recognize.
Also I agree on your comment on compiler work. I have a program that I launched in 1998 (NGS - Next Generation Software NSF01-147 http://nsf.gov/funding/pgm
16. Ray Hoare, Ray of Pitt <Hoare@engr.pitt.edu> agreed with overload, and added specialization:
I agree with the overload of choices argument and believe that we would be better served by some sort of classification of problem domains and then have more focused tools and fewer of them for each domain.
It seems to me that many languages are trying to solve the entire problem and not just a niche problem, which is much easier to do well and to do robustly. It would seem to me that we could learn something from Microsoft (heretic, heretic!) in their marketing. They have niche market tools such as their Active Server Pages tool for web pages (or JSP). They are easy to use but only work for web pages. However, if I'm working on dynamic web pages, I focus my choices on what is the most productive rather than determining the best language. What is not clear to me, is the categories of HPC applications and a way to enable multiple tools to work together to solve the larger HPC problems. Can't we classify HPC beyond shared memory and message passing? Has anyone done a classification of problem domains?
17. John D. McCalpin of AMD <jmccalpin@austin.rr.com> speaking for himself, recounted anecdotes and principles:
While boring everyone with anecdotes, I will try to roll up a few fundamental principles that seem to be demonstrated by the industry's experiences with parallel programming.
1. Many decisions do reduce to money, but not always in obvious ways.
· Resources have varying degrees of fungibility in different organizations. Academia seems to be the most restricted, followed by government labs, followed by large corporations. Resources are most fungible when considering the market as a whole (i.e., different startups may choose to allocated the same monetary resources in dramatically different ways). If an entire segment of the industry is not free to allocate its resources to maximize the attainment of its own goals, then the market as a whole will be inhibited from responding effectively.
· A good example is "productivity". DARPA is putting hundreds of millions of dollars into industrial R&D for the design of more "productive" systems for high-end computing. Speaking as a money-grubbing industrialist, I would rather that the government simply make its purchasing decisions based on their actual productivity metrics. This would enable industry to creatively allocate R&D resources across hardware, software, and consulting/collaboration to produce optimal solutions. Unfortunately, actual system purchases continue to be made on the basis of either throughput on existing workloads (not a bad metric at all, but it does ignore development productivity issues) or peak performance (or its LINPACK proxy). I am not in a position to understand the relative costs of application development productivity in government programs, but I seem to recall hearing that the ASC (nee ASCI) program has spent more money on software than on hardware. As an industrial participant in many of the ASC/ASCI procurements, my recollection is that the buying criteria were based on application throughput and uptime, not on development productivity. Moral: You get what you pay for.
· Of course even in (ostensibly) rational organizations different groups have different goals. The person(s) paying for the hardware may not be the person(s) paying for the application development, so the two groups would naturally have different perspectives on how to optimize the other group's decisions. :-)
2. Longevity has certainly scared people away from many programming languages. From an economic point of view the most important people in this respect are not the readers of this mailing list -- the most important people are the decision-makers at the Independent Software Vendors (ISVs) for scientific and technical applications. A very large fraction of the overall HPC server market revenue (and the majority of the profit) is associated with customers running ISV codes (or "community" codes with similar decision criteria). It took many years for ISVs to become convinced that MPI had enough longevity to serve as a target, and then many more years before a broad ensemble of fully-functional MPI implementations of these codes became available.
3. Portability was a primary factor in blocking SGI's effort to get scalable shared memory programming to displace MPI (which was still not strongly established in 1996 when SGI launched the Origin2000). Since no other vendors made scalable shared memory systems, customers were quite naturally nervous about porting to this environment -- no matter how appealing it might have appeared. The other primary factor that blocked scalable shared memory programming was a lack of performance transparency. We discovered too late that getting great performance on the system required that the user understand the algorithm choices to solve a problem, the implementation choices for those algorithms, the compiler's strengths and weaknesses, the operating systems policies for NUMA control, and the hardware characteristics of the system. Failure to make the correct choices in any of these areas often led to unsatisfactory performance, and very few people outside the SGI home office "performance engineering" team had access to the full range of that information.
· Lession: a system that delivers a high level of abstraction must also deliver tools to monitor and control the performance-related features of the implementation. Inside SGI the performance engineering team knew how things worked, so we were able to rapidly identify performance bottlenecks. Lacking that in-depth knowledge, customers needed effective performance tools that SGI did not deliver soon enough. ("Soon enough" should have been with the initial launch of the product -- first impressions of early users are very important.)
A good compiler, parallel language or not, is expensive to develop. The customers aren't satisfied with a mediocre one.
A good case in point is Fortran90. The first working compilers were not available until ~1994, and the first really good compilers were not available until sometime in the second half of the 90's. When I published a short paper in 1996 on the topic, only one of the compilers I tested was tolerable. The lack of good Fortran90 compilers is certainly a major factor in the demise of HPF. HPF has/had other flaws, but I think it would have been "good enough" for many application areas if it had better compiler technology behind it. By the time the good compiler technology was available, the industry had moved on....
Horst Simon said:
But there is more to it. Actually the NRC report on the "The Future of Supercomputing" (see http://www.nap.edu/catalog
This is of course a very important lesson, but it misses one crucial point -- we have ten years experience in building applications that make sense to build on MPI. But we have relatively little documentation of what sorts of applications might be enabled if the development cost overhead of MPI and the fundamentally synchronous model of programming of MPI could be avoided. Of course a large part of the motivation behind the DARPA HPCS program comes from people who do not believe that they are well served by the clusters that currently dominate the market.
A final observation: While hardware continues to get cheaper, software is bifurcating:
· If you want to do something that can be done with open source software, or with a "framework" that someone else has paid for (e.g., many really interesting systems coming out of the ASC program), then software can be considered relatively cheap. The set of activities that can be done in this context is growing rapidly.
· If you want to do something that cannot be done with open source software (or with someone else's "framework"), then software is becoming prohibitively expensive.
This drives development to higher-level languages, which are difficult to optimize even on uniprocessor platforms. Parallelization of high-level languages requires a coherent "ecosystem" of user-centric performance and debugging tools. Unfortunately no one wants to pay for these because using the high-level language was an attempt to avoid cost in the first place!
18. David DiNucci (above) then disagreed with Horst Simon:
I think the "plethora of choices" argument is, to some extent, a red herring. Nobody would be complaining if just one or two of this plethora were exemplary. Word of mouth would quickly bring those to light. The real issue is not so much the number of choices as their overall lack of utility or usability. Many are underfunded or defunded research projects that are being exposed to the world in hopes that someone will find the "good parts" useful, rather than just throw years of work into the trash for lack of commercial viability.
So I would assert that this unappetizing menu of projects is just another symptom of funding difficulties. If there's no market for products past MPI, funding agencies should not even pretend to be interested in funding them. If there is a market, or it's a chicken-and-egg issue where the market can't form until the product exists (bingo), then funding for these products should be planned up front to continue on all fronts until there's at least a chance of commercial viability, for the sake of both the funder and fundee. We need more productive ways of filtering and integrating promising technologies into a few efforts worth such major funding, and extra care must be taken to avoid conflicts of interest in these funding decisions.
EA's recent 5% layoffs have been partially attributed to difficulties in transitioning to the new multicore game machines. This is just one example of a market where MPI will likely not come to the rescue. And games are just the leading edge of multicore.
19. Greg Pfister (above) posted this summary of what had been said so far:
Note, all is his own opinion, not IBM’s.
|
$$$$: cost of compilers + application portability and longevity. |
Greg Pfister, IBM |
|
People don’t want new language. Low quality & interoperability. |
Alan Edelman, MIT |
|
No benefit; embarrassingly parallel, or not worth it. (Amdahl). |
Rich Enbody, MSU |
|
Catch-22: Won’t use unless popular, not popular unless used. |
Larry Rudolph, MIT |
|
Low quality compilers and object code. Portable ≠ performant. Resistant: don’t know new language, but it’ll be called Fortran” |
Allan Gottlieb, NYU |
|
Agree with $ reason; compilers need iterations to get right. |
Marc Snir, UIUC (CS Dept head) |
|
Situation will change because of new Microsoft interest. |
Rajkumar Buyya, U. Melbourne |
|
FSS likes array languages; programs do what they think they do. |
Robert Bernecky, rogers.com |
|
Yes $, portability, longevity. 99% compatible not good enough. |
Niraj Srivastava, HP |
|
Yes $, but HPC is ecosystem, portability just one link; all must move together. See NRC report. |
Horst Simon, LBL Assoc Lab Director CS |
|
Agree with Horst. MPI works. Low cluster cost + MPI portability discourages anything but clusters. Not good. |
Duncan A. Buell, U. S. Carolina (Interim Dean Eng’g & IT) |
|
Choice overload (psych ref). Too many = don’t pick any. |
Timothy Mattson, Intel |
|
David C. DiNucci, Elepar |
|
|
Low performance; performance is the only reason for parallel. |
Srinivas Aluru, IA State, assoc chair ECE, chair Bioinformatics |
|
Agree with compiler $ reason. Funded res. program, future results. |
Frederica Darema, NSF |
|
Agree with choice overload. No GP parallel lang.; need target.. |
Ray Hoare, U. of Pittsburg |
|
Longevity issue scares ISVs. Portability hurt SGI. No good F90 compiler until 1994. We don’t know what apps MPI can’t do. |
John D. McCalpin, AMD (speaking for [him]self...) |
And later boiled everything down even further.
Why Static, Nothing but MPI? Summary:
· Application longevity and portability.
· Apps outlive hardware. Good compilers are expensive to develop. Can’t presume anything but broad standards will always be available. (Me; repeated agreement.)
· The longevity issue particularly scares ISVs. (John D. McCalpin, his SGI experience)
· Languages are just one link in an ecosystem. All links needed for success.
· “the perfect parallel programming language will not succeed [without, at introduction] effective compilers on a wide range of platforms,… massive support for re-writing applications,” analysis, debugging, databases, etc. (Horst Simon, LBL Assoc Lab Director CS; from 2004 NRC report)
· Users are highly averse to using new languages.
· “I don't know what the dominant language for scientific programming will be in 10 years, but I do know it will be called Fortran” (quoted, Allan Gottlieb, NYU)
· Catch-22: No use unless popular, not popular unless used. (Edelman, MIT)
· High quality object code required at introduction; seldom the case. (several)
· The only point of parallelism is performance.
So regarding parallel languages and their history, he concludes:
· This is seriously well-trodden ground, a very popular research area. 100s of languages have come and gone.
· Good stuff must have been created – by pure brownian motion, if nothing else.
· Yet it is broadly accepted in the field that they’re not used.
· MPI has a lock.
· OpenMP is accepted, but a distant second.
· There are substantial barriers to the introduction of new languages and language constructs.
· economic, ecosystem related, psychological, a catch-22 of widespread use, etc.
· Any parallel language proposal must come equipped with reasons why it will overcome those barriers.
