Grand Central Now Open to All

Author: Drew McCormack

Apple today made the source code of Grand Central Dispatch available under an Apache open source license. One of the new technologies for concurrency added to Mac OS X 10.6 Snow Leopard, Grand Central is Apple’s attempt to help developers deal with the rise of multi-core. The open sourcing of Grand Central comes as something of a surprise, because it is a core technology in Snow Leopard, and could be seen to give Apple a competitive edge in the new world of multi-core.

So why did they do it? Only Apple knows for sure, but there are compelling arguments for open sourcing Grand Central Dispatch, even for a commercial enterprise. First, Apple will of course reap the rewards of any development that takes place, just as they have with WebKit. Second, it is unlikely that Grand Central would be used by any direct competitor to Apple, like Microsoft. Grand Central is more likely to be added to other UNIX and Linux systems, none of which really pose a threat to Apple’s consumer-based business.

This leads to what is perhaps a more important consideration for Apple, that allowing Grand Central to be ported to other UNIX/Linux systems will encourage its use. Until today, it would have been very unlikely that any new UNIX tools would be developed on Mac OS X using Grand Central, simply because they would only run on the Mac. With the possibility that Grand Central will become available on other UNIX systems, the likelihood that Grand Central will be incorporated into command line tools is greatly increased.

Of course, this is also very interesting for scientific developers. It may be possible to parallelize code in the not too distant future using Grand Central Dispatch, and run that code not only on Macs, but also on clusters and supercomputers.

There could be one last reason why Apple has taken this step: they want to use Grand Central to push the adoption of other technologies, in particular, blocks. Blocks are an extension to C which form the basis of Grand Central Dispatch. Having your operating system based on a non-standard language is not a good position to be in, and Apple would surely like to see blocks incorporated into the C language. By offering Grand Central to the broader programming community, they may be hoping it will catch on, and make the argument for incorporating blocks in the C standard that much stronger.

All of this is in the future, and it may be some time before any of it comes to be. For a start, the Grand Central Dispatch code that has been released is only the developer APIs, not the kernel support, which would need to be rewritten for each platform. C compilers that support blocks are also in short supply, though presumably Apple will release its changes to GCC at some point, as required by the GPL.

In any case, this is an exciting move, particularly for scientific developers. If Grand Central Dispatch does catch on, it could provide a genuine option for code parallelization on and off the Mac.

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Kernel support is available

From the libdispatch page linked above, it looks like they've made the source for kernel support of GCD available in their XNU sources. If you don't want to wait for GCC to support blocks, there's always LLVM.

Looking at this from the perspective of broadening support, while not enabling their largest competition (Windows), this makes sense. Look at OpenCL, LLVM, and Clang. All three of these are great new technologies at the heart of Snow Leopard, and none of them will be embraced by Microsoft. However, the Linux community sounds pretty excited about getting to use these in their applications, and even people at Google I've talked to sound interested in what OpenCL might be able to do in their server farms. I know I'm looking forward to the Cell implementation of OpenCL to make programming for the PS3 architecture tolerable.

Microsoft's aversion to open standards (DirectCompute vs. OpenCL, IE vs. WebKit, etc.) may end up putting them at a disadvantage if the rest of the world starts using these technologies.

_______________
Brad Larson, Ph.D.
Sunset Lake Software ( www.sunsetlakesoftware.com )

Glad to see this

Yes, I'm very glad to see that Apple is making GCD and blocks into some form of open source. I believe that makes its adoption far more likely. Without worrying about code that might only run on the Mac, a writer can more easily see that GCD/libdispatch is the way threads "should have been". I would underscore Drew's statement about applicability to scientific developers, since we often need computational codes to work cross platform (even when Mac is the favorite). The benefits of the GCD/libdispatch technology can be adopted unhampered. Thanks Drew!

Why?

I doubt that Apple can help researchers. GCD is a thread pool which interacts deeply with the kernel. No big deal. Not exciting from a scientific point of view. Have you ever heard of TBB???

I think Apple wants gcc and linux kernel to support GCD, that's the real reason for making it open-source. It's marketing.

I seriously hope Linus will oppose to it.

Why Not?

GCD is a shared thread pool that is balanced between all running applications utilizing it. From what I understand of TBB, its model is to make threading easy within your application, not across all of them. The kernel should have to do far less context switching with GCD.

Apple doesn't need GCC to support closures closures - it seems pretty clear that LLVM is the future for all of their compiler needs. However, they're extremely proud of their new tech, and want to get it out there while it's only one of the few solutions to this problem; probably to be standardized. Nothing's worse than a fractured developer community.

GCC with Blocks support is available

How else would Plausible Blocks work? http://code.google.com/p/plblocks/

GCC with Blocks

Blocks are in Apple's GCC branch, but not in the main GCC repository, as far as I know.

Drew

---------------------------
Drew McCormack
http://www.mentalfaculty.com
http://www.macanics.net
http://www.macresearch.org

The paranoia of open source

"I seriously hope Linus will oppose to it."

The paranoia of open source fanatics never ceases to amaze. Who ever would have predicted that a communicy that allegedly values openness would have such a chronic case of Not Invented Here syndrome?

Apple and Open Source

Apple is doing the right thing here, and has been consistent in their approach to open-source vs closed-source. They seem to really want to keep the kernel + foundations of their OS open-source, even when they add completely new things, including GCD. That's the same approach they took with launchd. They also embrace and extend existing tools, becoming the main contributor, as with WebKit and LLVM. They don't give away very much by open-sourcing these tools and it helps keep the ties stronger with the Unix-side of OS X. Their competitive advantage is in the higher levels of the OS and in the overall integration of its parts.

I am not sure contributions to the source code outside of Apple really are happening very much, but it definitely helps with their image and in pushing standards that fits their plans.

Re: Why?

It's stunning how a company can develop a technology and give it away for free, yet someone always find a way to complain about it or claim that "there's nothing new here". As Drew illustrates in his follow-on to this, there are significant performance benefits to be had by using GCD to perform operations that can be done in parallel. Many, many scientific applications can take advantage of this, and the easy system-wide load balancing provided by GCD makes it practical to get the most out of your system.

I've been rewriting my Cocoa applications to use NSOperations and NSOperationQueues instead of manual threads, and seeing significant performance boosts. On Snow Leopard, all NSOperations are backed by GCD, so you get an additional speed improvement for free. On top of all this, operations and queues provide for cleaner multithreaded code.

As far as your opposition to Linux incorporating a technology like this at the kernel level, that just smacks of zealotry. It's not just marketing, this is something real that works well to let us access the full power of modern computers. Why not integrate that into more systems?

_______________
Brad Larson, Ph.D.
Sunset Lake Software ( www.sunsetlakesoftware.com )

Re: Why?

Heh, Intel demo'ed TBB during a session at the last WWDC; they then ugly-hacked GCD onto the same demo app, and it *performed better* than the TBB based app. Intel's library and Intel's own demo versus a GCD demo written by Intel, and GCD won.

I also don't see how you could object to someone making a GCD kernel module for linux. How is that a bad thing?

Apache License version 2

The Apache License is a pretty good license for something that one wants wide adoption of.

I can't comment on the technology, but the license sounds pretty good. I *hope* the technology is good, and soon included in GCC, because with the increasing prevalence of multi-processor X multi-core computers SOME solution is needed, and the Apache License is probably just right for this.

Mac OS X Enhancements

Now only if Apple would port over the following:

Security Enhancements:
FreeBSD Jails: http://www.freebsd.org/doc/en/books/handbook/jails.html
PF: http://www.openbsd.org/faq/pf/

Filesystem:
ZFS/Lustre FS: http://www.redorbit.com/news/technology/99870 7/clu...
HammerFS: http://www.ntecs.de/blog/articles/2008/01/ 17/zfs-v...
GitFS: http://www.sfgoth.com/~mitch/linux/gitfs/
ProcFS: http://en.wikipedia.org/wiki/Procfs

Add OpenMeta (http://code.google.com/p/openmeta/) to:

Finder: http://ironicsoftware.com/leap/index.html
Spotlight: http://ironicsoftware.com/deep/index.html

Good News -- Many Eyes; More Confidence; Broader Adoption

This is good news for everybody who cares about this class of technology, i.e., almost everybody involved with scientific software.

GCD's objective is to make it relatively easy for scientists, whose expertise is not in tweaking software for parallelism, to amp up the speed of their programs. And by “easy,” I mean also “confident,” “informed,” “productive” and “efficient.” If this approach works well on multiple platforms -- if the requisite “code blocks” are adopted into GCC and other compilers; if runtimes are available on linux, Windows and maybe even specialized, high-throughput OSs -- scientists and developers are much more likely to adopt it without worrying about locking into this year's nifty technology platform. Also, any implementation shortcomings can be patched up with more eyes on it, and third-party books, etc., can show authors/engineers how to make the most of it.

Of course, it's not incidental that MSFT is releasing an incompatible technology with the same objective of speeding up (Windows only) programs in the near future. Apple must have made a calculation that to get the best support for its technologies, it would price it right, making it widely available, to win the Mind-space battle. Probably very self-interested (as a shareholder, I hope so!) but still, a win-win if it's as helpful as it looks.

“Inquiring Minds Want to Know!”

Generalizations

"The paranoia of open source fanatics never ceases to amaze. Who ever would have predicted that a community that allegedly values openness would have such a chronic case of Not Invented Here syndrome?"

I don't want to turn this into a Internet cat-fight, but I would discourage you from inferring the beliefs of a community from the "loudest" people. ("Loudness" in the context of the web is the probability of a person hitting the "submit" button on a comment form.) Postings on the web are not an unbiased sample. People with strong views (both rational and frequently otherwise) are more likely to be seen.

The paranoia of paranoids afraid of open source....

Quote:
"The paranoia of open source fanatics never ceases to amaze. Who ever would have predicted that a communicy that allegedly values openness would have such a chronic case of Not Invented Here syndrome?"

user/member dzd extrapolates from the reaction of one person who stated their preferences and assumes that those stated personal preferences reflect the attitude of an entire community.

Is this a reasonable extrapolation? No. One data point has an infinite number of slopes. (or, technically, zero slopes :-) )

Member dzd's reaction to that stated preference, smearing the entire open community, is a sad commentary on what passes for rationally thought out analysis in society today.

One person does not speak for the entire community. Hell, even Linus doesn't speak for the entire community. [But he for darn sure has a lot of influence! :-) ]

Its amazingly similar to the "If yer not with us, yer agin' us, yer with the terrists!" thinking promulgated by recent (and also less than rational ) political stupidities.

My suspicion is that if the code is good and the idea is scalable, extensible and plays well with the existing kernel features, that it stands an excellent chance of being incorporated into the kernel. My best advice? Get the code to be sponsored by one of the current Kernel insiders. That way it will be seriously evaluated for its true merits. ( see the Kolvas(sp?) scheduler scenario)

I won't be returning to this thread as I joined specifically for the purpose of responding to that comment.

People, Please don't give in to the urge to respond emotionally without thinking first. We are not politicians emotionally manipulating a crowd of mouth breathing religionists. THINK!

Re: Why?

what makes you apple wants it in the linux kernel? Or that it even has to go into the kernel to see support? If things like QTConcurrent move to using a global pool of threads instead of a local pool, that could be a huge boost for QT apps on the desktop and not be implemented in the kernel any more than QTConcurrent is in the linux kernel. And since QT/KDE runs cross-platform now, their mac version could even just use GCD as the backing, gaining the benefits of OS X optimizations that Apple has made for its platform.

Linus, or any kernel dev, doesn't have anything to do with this showing up if people in userland want it.