Announcing OpenMacGrid: Together We Are Strong

MacResearch is proud to announce a new project that draws upon the very best of Mac OS X and the Mac-using scientific community: OpenMacGrid. As the name suggests, OpenMacGrid is a computing grid built up entirely of Macs, and open to Mac-using scientists everywhere. OpenMacGrid is based on Apple's Xgrid technology, which comes pre-installed with every copy of Mac OS X 10.4 (or later). The potential of Xgrid has always been obvious, but until now, it has never been fully realized. OpenMacGrid will change that, by bringing the spare cycles of Mac users, together with the computational requirements of Mac researchers.

Open to All

Projects that harvest the spare cycles of commodity hardware are nothing new. It all began with SETI@home, and there are now multitudinous different causes that you can donate your computer's idle time to, from protein folding to relativistic physics. Noble as these projects are, they all share one limitation: they are closed networks. Each of these grids runs a single application, and they are not available as a computational resource to outside scientists.

Think Different

OpenMacGrid is different. Like the other grids, Mac users will be able to donate their idle time to challenging scientific computations, but unlike the other networks, researchers will be able to access this resource with their own scientific applications. The programs running on OpenMacGrid will vary from day-to-day, reflecting the projects being undertaken by Mac researchers at that point in time. In this way, Mac users can directly aid scientists in achieving important scientific breakthroughs.

A Lot of Horsepower

If you're thinking OpenMacGrid will be an insignificant computational resource, think again. Testing has been underway for some time, and even before allowing the general community to connect up their Macs as Xgrid agents, a study of gravitational waves has been undertaken by our own Gaurav Khanna. Gaurav's calculations peaked at around 200Ghz of CPU power. This will increase considerably as the public connects up their machines, making OpenMacGrid a serious computational proposition for Mac researchers.

Participate

We begin the OpenMacGrid rollout today by inviting you to connect your Mac to the OpenMacGrid controller. This requires no more than entering a few details in the Sharing pane of System Preferences, and is described here. After you connect up your Mac, you will probably want to download the OpenMacGrid widget — based on the Xgrid@Stanford widget created by Mekentosj and Charles Parnot — so that you can monitor OpenMacGrid usage, and see when your machine is contributing.

Today we are also opening the grid up to scientists. This will work as follows: If you have a project you would like to run on OpenMacGrid, you can fill out an access request form and submit it to MacResearch. If there is adequate capacity available on OpenMacGrid, and your proposal meets the criteria, you will be allowed to submit jobs to the grid. MacResearch will judge whether projects are suitable for OpenMacGrid based on their system requirements, and other criteria, to prevent overburdening the agent machines that have been donated.

OpenMacGrid is an important initiative for MacResearch in 2007 and beyond. Hosting the world's largest Xgrid network, and likely one of the largest open grids in the world, fits very well with our reason for being: the facilitation of scientific research on the Mac. We are very proud of the initiative, and with your help we can all make it something ... dare I say it ... insanely great!


Contribute to OpenMacGrid
Download the Widget
Submit a proposal for access

-----------------------------------------------------------------

DISCLAIMER:

By contributing an agent to OpenMacGrid you recognize that third parties will be running software applications on your system. MacResearch.org, will make every attempt to ensure that third party applications are safe for execution on contributed systems (agents), but in no way will MacResearch.org or its affiliates be held liable for any damage to your system resulting from participation in OpenMacGrid. It is the responsibility of the person contributing the agent to ensure that they have permission to use the system in such a fashion.

By submitting a computation project, you recognize that your software will be running on third party systems. The potential, therefore, exists that a third party will be able to obtain your application and run it independent of the project or in a manner not intended for the original use. Should this occur you agree that MacResearch.org will not be held liable for any damage or loss of intellectual property.

OPENMACGRID IS PROVIDED BY MACRESEARCH.ORG ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL MACRESEARCH.ORG OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF OPENMACGRID, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Congratulations!

This really looks to be a promising, and even revolutionary, project. Scientists (including graduate students like myself) are increasingly encountering problems requiring grid-computing capabilities. And not all of us have either the resources or capabilities to create and distribute proprietary closed-network software like Seti@Home or Folding@Home, etc.

I've signed up and encouraged others to as well. Hopefully someday soon I'll be sending in a proposal for time on the grid!

A new Xgrid feature needed

I would love to add the grid of ~50 nodes I administer to the OpenMacGrid project. However, some of our users need Kerberos authentication, and many would be upset to have their jobs superseded by jobs from out-of-institution projects. For both these reasons, we need some form of nested Xgrid topology... for example, a setting in the Xgrid controller that would let it accept jobs from an other controller if there were no jobs in its queue would be a start. Then our controller could accept OpenMacGrid jobs when there weren't in-hosue jobs waiting. Hopefully the existence of OpenMacGrid will help Apple prioritize something like this.

See what I'm contributing to...

I just signed up! It would be nice to be able to see what projects my Macs have contributed to.

-Sean

Seeing what is running...

We have a few ideas about this, but nothing concrete yet. We'll keep you all posted.
Thanks to everyone who has signed up so far. The interest was a bit overwhelming, but is getting under control now.

Anyone out there thinking about submitting a project, no matter how big or small, please do. We have a lot of capacity now, and we need some people to start using it.

Drew

---------------------------
Drew McCormack
http://www.macanics.net
http://www.macresearch.org

Great project!

This is really a nice idea!

Is there a way to find out the intel/ppc cpu ratio of contributers to the grid? I personnaly run code that use the intel MKL library that obviously does not support the ppc architecture so this kind of information would be quite interesting (to me at least!).

-Chuck

Intel vs PPC

Not that I'm away of. If you want to run on OpenMacGrid, for the time being you would need to create a PPC version with a non-MKL library, and use MKL for your intel version.

Who knows what will be possible in Leopard though...

Drew

---------------------------
Drew McCormack
http://www.macanics.net
http://www.macresearch.org

Nested Xgrids

I don't think Kerberos would be a serious option for OpenMacGrid, I'm afraid. A nested controller architecture would be welcome, but this is probably something Apple needs to do.
Note that there is a little bit of this functionality already: you can create 'grids' on a single controller. If your people wanted control over their machines, they could create a separate grid on the OpenMacGrid controller. Most people will just submit to the default grid.

Drew

---------------------------
Drew McCormack
http://www.macanics.net
http://www.macresearch.org

Xgrid Projects of all Shapes and Sizes Welcome

You are certainly welcome to submit a proposal. Anything reasonable will be accepted, big or small.

Drew

---------------------------
Drew McCormack
http://www.macanics.net
http://www.macresearch.org

Xgrid in Leopard

I thought that Xgrid 2 would allow a person to make all sorts of decisions about which agents to use or not use based on all sorts of hardware criteria. Is that not correct? Of course that doesn't mean that you guys would have been able to build such a feature into OpenMacGrid yet, but if I remember all my wild rumors/speculations correctly then it shouldn't be impossible.
JM

What about BOINC?

Will it be possible to run any sort of BOINC project (SETI@home, Folding@home, etc.) through OpenMacGrid? I would really like to contribute to already large projects instead of small ones.

Ciao,
Nicolò

What about BOINC?

We are working on integrating BOINC at the moment, and will probably use it to fill OpenMacGrid when our own projects are not filling it.

But I'm not sure I agree with donating only to 'already large projects'. Many of them use enormous resources for very little result (not naming names). I actually think you are better off betting on many smaller projects.

Science, by its nature, is unpredictable. Discoveries come from unexpected places. Spread your bets. Putting all your resources on a 50 to 1 long shot could pay off hugely, but more than likely will put you in the poor house.

Drew

---------------------------
Drew McCormack
http://www.macanics.net
http://www.macresearch.org

Is it working?

Hi, I just signed up but its not clear to me that my mac is doing any work (cpu % is always very low). How can I tell?

Channing

I agree :-)

You're right, I'd be happy to spread my bets. I'm looking forward to more info about what I'm betting on.
:-)

Thanks for trying to integrate BOINC anyway, I need to use that argument to convince other Mac people to join the grid.
:-P

I would also be interested in knowing if, by any chance, there's some opportunity to choose among projects I want to run on my system. For example, I might be interested in running only biology-related projects (large and small, in order to spread my bets), because, for example, I might not be interested in advancing the human knowledge on, say, mathematics, because I think that should be given lower priority at the moment than biology. Is any sort of differentiation possible or will we be able to just join the grid or not join it?

Ciao,
Nicolò

Firewall?

Since this is a really good idea, I have set up my Mac mini server to accept jobs (my MacBook Pro is not yet in the swarm...). Now I wonder if there is really nothing I have to do with my firewall, I couldn't find any info related to that. Anyone knows something?

Is OpenMacGrid running on my computer?

If you install the widget, and the light is 'green', your computer is running a job.
But it will not always be running things, so if it is yellow, it doesn't mean there is something wrong.

Drew

---------------------------
Drew McCormack
http://www.macanics.net
http://www.macresearch.org

Choosing Projects to Run

This is an interesting idea, and it may become possible when leopard comes out, but for now it is not trivial. At the moment, if you join OpenMacGrid, I'm afraid you will have to settle for helping all scientists, rather than just biologists or physicists or ...

Drew

---------------------------
Drew McCormack
http://www.macanics.net
http://www.macresearch.org

Not Discussing At-the-moment

NDA

Drew

---------------------------
Drew McCormack
http://www.macanics.net
http://www.macresearch.org

Do I need to open my firewall for OpenMacGrid?

From the Xgrid FAQ at Apple (http://lists.apple.com/faq/pub/xgrid_users/):

"That depends. The only component of Xgrid which has an open port -- and thus requires a firewall opening -- is the Xgrid controller, which listens on port 4111. This also means the controller is the only component which advertises on (i.e., responds to queries over) mDNS. When enabling the Xgrid controller, make sure that firewall port 4111 is open on your computer's firewall (if enabled, in the Sharing Pane of System Preferences) or your corporate firewall (if accepting agents or clients from outside your organization).

Agents and clients simply hook up the controller via either an mDNS lookup or an explicit hostname/IP address, then initiate a new BEEP connection to the controller over a user port, thus requiring neither privileged operation nor special firewall settings.
Last update: 2004-11-23 14:09
Author: Ernie Prabhakar"

Drew

---------------------------
Drew McCormack
http://www.macanics.net
http://www.macresearch.org

Client

Ah, thank you. If I think about it, that's actually quite obvious; my Mac acts merely as a client and obviously fetches his tasks from the controller, it's not that the controller sends something (unrequested) to my Mac.

Thank you for the link.

detailed stats?

after signing up with the website and adding my mac to the grid, how can i see specific detailed work on what my mac has done? point, etc?

Detailed stats

This has been a common request. At this point, we don't have access to those type of stats, I'm afraid. We are considering how we could do this in the future, so stayed tuned...

Drew

---------------------------
Drew McCormack
http://www.macanics.net
http://www.macresearch.org

Older macs feel left out

I have serveral older G3 PowerMacs that only support 10.3.
Is there any way they could be used for the OpenMacGrid?
I saw the requirements were 10.4.
I know there is an Xgrid client for 10.3 that allows it to be connected to an Xgrid on 10.4

OpenMacGrid on 10.3

Yes, you can donate your cycles on 10.3 by downloading the Xgrid agent:

http://www.apple.com/support/downloads/xgridagentformacosx103.html

But as far as I know, you can't submit jobs --- there is no client for 10.3.

Drew

---------------------------
Drew McCormack
http://www.macanics.net
http://www.macresearch.org