Five Minutes with Apple
By drewmccormack at Wed, Jan 23 2008 2:26am |
If you had five minutes with the leadership of Apple's Science division, what would you ask? What are the biggest open questions for Science on the Mac? And what should Apple do to get more/better scientific apps to the Mac? If you have a view, let us know in the comments.
(This request comes from Attila Csordas, who may have the opportunity to pose some of these questions for real in the near future.)
Comments
Pages should work with bibliography software
Scientists have to write grants and papers that need bibliographies. Pages is a great alternative to Word, but does not support EndNote, Bookends or other bibliography software. Why not?
Scripting
It is great that Apple includes Python and Ruby in Mac OS X. Really nice is that Apple included some science oriented packages with the python install (numpy). What is not so great is that the versions are fairly old, and because they are installed in /System, updating becomes hard. This is the reason the TeX community choose to provide an installer themselves: the 20 MB provided by Apple would have been insufficient, and the burden of updating would have been significant. Some publicly discussed ideas on how to proceed would be nice.
With leopard Apple made a mistake in Python: the distutils are using a wrong deployment setting, making it impossible to install mixed code python packages. A correction in 10.5.2 would be good.
Maarten
Re: Scripting
Hi Maarten,
You have some fairly specific complaints here, but what is your view on the big picture of what Apple is doing? You seem to be implying from your post that you would rather Apple not ship scientific packages with the OS, but instead let the community provide them. Is that right?
Drew
---------------------------
Drew McCormack
http://www.maccoremac.com
http://www.macanics.net
http://www.macresearch.org
GPU computing using CUDA
A number of folks have been waiting for an Apple/NVIDIA driver that supports CUDA. I understand Apple's reluctance to support single-vendor solutions, but please at least give us the ability to make that call ourselves. If we want NVIDIA-only solutions, don't hinder our choice. I would love to build a GPU-cluster using Xserves and NVIDIA Tesla rack-mounted GPUs. As it is now, I'm forced to pursue non-Apple options.
Pages, MATLAB, and LaTeX font cache corruption
1. I agree with the earlier commenter that Pages should play nice with reference software. While we're at it, some sort of support for equations would be nice too, even if it is only easy numbering and referencing to go with equations exported from LaTeXiT or similar.
2. The MathWorks needs to get MATLAB out of X11 and into the real Mac world. Perhaps Apple could lean on them?
3. A fix for the ongoing problem with LaTeX font-cache corruption in Leopard would be greatly appreciated (see here: http://www.macresearch.org/fixing-broken-latex-fonts-leopard).
- Chris MacMinn
Scripting
Well, that is a delicate one: I really like that you can count on a base version being there. For scientific use, you'll need to install quite a few additional packages: numpy and scipy are probably needed for all, matplotlib is pretty basic as well. On the one hand I'd like Apple to install these, but offer a method of updating them later on. So it is not so much: "don't supply, we can do better", as was the decision with TeX.
It is more: "Install, please, but not in /System." Later on we can discuss how the Apple installed software could be kept more up to date, and what additional packages to install. Moving from /System to /Library would be a good first step, I think. It really is a philosophical step: I really like that Apple embraces these new and open systems. It adds value to the system. But right now the first thing I have to do to get my additional packages up and running is installing the python install package from Python.org. There must be a better way.
Maarten
Same with IDL
IDL by ITTVIS is a scientific data analysis package that lives in X11, and not a cheap utility (same with MATLAB). I doubt though that leaning on either of them will produce a non-X11 version of their products. They'll probably cite market share and ROI on that. Apple talking with them can never hurt though. I'm not holding my breath, and slowly move over to python.
Maarten
Java, Xcode, X11
Faster adoption of new Java standards-for more than ADC members.
The terminal to be X11 compliant. Instead of having to install a separate X11 system.
Code folding and Focus follows selection for free format Fortran.
Big Picture: Scripting, Plugins and Extensibility
I think one "big picture" question centers on scripting, plugins, and extensibility.
Some Apple programs support a documented plugin API. Many programs support Apple Scripting (and now Python and Ruby as well). But this does not cover all programs from Apple, much less third-party programs.
Many scientists, like myself, see good design in Apple programs, but "if only" a certain feature was added. Case in point, in iWork, Numbers does not have plugins or script support. Users would love to see integration between Pages and reference software. (This might be possible with AppleScript, but doesn't exist yet.)
What are Apple's plans with respect to extensibility and scriptability? Will all Apple programs have AppleScript support by the end of 2008? Will more programs have plugin interfaces?
slowly moving over to Python...
I, too, have been considering a move to Python - and I know that I am not alone, even within my department. Perhaps the potential of defection to Python/SciPy will eventually convince these developers that X11 as a "good enough" solution for Mac is just not good enough... especially because if such a shift reaches critical mass, it could easily propagate into their PC user base.
Of course, I'm not promising that a native version of MATLAB would stop me from moving to Python anyway... it does have other advantages.
- Chris
gfortran: install by default?
The GCC compiler suite is about to launch version 4.3.0. One of its many changes is:
"Due to the fact that the GMP and MPFR libraries are required for all languages, Fortran is no longer special in this regard and is available by default."
http://gcc.gnu.org/gcc-4.3/changes.html
It is known that Apple has developer preview builds of GCC 4.2 for OS X. Will Apple, for that GCC 4.2 suite or a future GCC 4.3 for OS X (perhaps even OS X 10.6), install gfortran by default with the Apple-distributed GCC/Xcode?
It should be relatively trouble free, and would make quite a few scientists happy.
pdf on the iPhone / iPod touch
Maybe one of macresearch's recent post on reading pdf on an iPhone or iPod touch might be relevant if this discussion with Apple goes beyond just the Mac and Mac OS X.
Re: Java, Xcode, X11
You are aware that on leopard (if configured correctly) X11 will start automatically whenever an application is started that load the X11 library? Do not double click X11 in the finder, just start xdvi from the Apple Terminal and watch the magic happen.
You may need to update your X11 with the current xquartz, and make sure that the $DISPLAY variable is not set explicitly, and check that .MacOSX/environment.plist file as well. If it looks like a socket in /tmp (/tmp/launch-8eyzSV/:0) then you're set.
Maarten
Re: gfortran: install by default?
Agreed. While they are at it: just make the whole of gcc available (including gjc although sun will certainly not like that).
Maarten
apple version of gfortran
I would love to see apple-gcc include gfortran in their suite, even if not by default, but available through the installer. I can say that getting all of the compilers and packages I have running requires a lot of effort and having a standard gfortran compiler to expect on the platform would be great. This also ties into the move to python that a lot of people are making as scipy and numpy require an apple gfortran compiler to make these packages work.
Should Apple Include Sci Tools?
One thing that a lot of people are bringing up is the inclusion of Scientific libraries and tools in Mac OS X. Stuff like python modules, tex, gfortran etc. My question is: Is it really desirable for Apple to do this?
A lot of these tools change very often, and need to be updated often, and there will often be situations in which the version included by Apple is unsuitable. (An example from my own experience is OpenMPI in leopard, which doesn't have Fortran bindings for gfortran, which means I have install my own version anyway.) That, and the fact that only a small percentage of Apple users actually need these things, make scientific tools/libs undesirable as part of Apple's core OS.
Would it not be better if MR made some sort of uber-package containing all the most important stuff in one installer, and updated that every 3 months. Stuff like tex, gfortran, scipy, etc. This would mean users would effectively be wasting some disk space, because you couldn't pick and choose packages (or maybe you could), but you would sacrifice that for easy of maintenance and installation. Those that would want to go with individual packages could always do that, of course.
Would such a simple installation be an advantage over the usual fink/macports approach, or wouldn't it make much difference?
Drew
---------------------------
Drew McCormack
http://www.maccoremac.com
http://www.macanics.net
http://www.macresearch.org
Does Apple really care about large HPC environments?
I work in an academic supercomputing center. While almost all of us use Apple laptops, our cluster is Linux with no chance for Apple to get their foot in the door unless they (not necessarily in this order):
1. Introduce a viable storage array. Xserve RAID with its' lack of SATA drives and redundant controllers not only isn't an option, it's a joke.
2. Work with someone to port their parallel filesystem to MacOS X. Xsan isn't anywhere close to robust enough for even a medium sized (250+ nodes) cluster.
3. Work with organizations like the Open Science Grid and CMS to get their software ported to MacOS X (or create some sort of Linux compatibility layer so that commonly used software written for Linux can run on MacOS X - possibly in conjunction with the next item).
4. Create a version of MacOS X Server customized for HPC. Strip out the stuff we don't need and make sure we have everything we do need (including stuff like Lapack, Atlas, OpenMPI / MPICH2) included and optimized for Apple hardware.
From a hardware perspective, the new 8 core Xserve looks like a really hot machine. Unfortunately, unless Apple addresses all of the above we'll never even bother to demo one. Personally, I find that to be a shame, as I really like most of Apple's products, so I'd ask if any of the above issues are ever going to be addressed...
Kevin
What Apple needs to do!
I think Apple has many areas where it can improve their offering for scientists.
- First Apple really needs to bring new Xserve Raid. Why is Apple ignoring it while still bringing new and faster Xserves? Xserve and Xserve Raid should evolve at the same tine, isn't it? The architecture of the xServe Raid is many years old, they need to bring SATA, and SAS, more fiber channels ports, etc.... For the HPC market Apple needs to have new machines with the same competitive prices that it has today.
- XSan has a formidable potential for HPC, they need to promote it more.
- With Leopard, Apple has brought Open MPI to OS X by default, but they need to do the same thing with fortran. A lot of people won't find any use of the OS X built in Open MPI without the support of fortran. They need to build fortran with Apple's gcc.
- Apple needs seriously to convince some big names in finite-element software (Ansys/Fluent, Dassault Simula, CFD++, Flow3D, MSC Marc-Nastran, LS-Dyna, etc) to bring their solutions to the mac. It is still too few on the mac platform. Leopard is now UNIX O3 compliant, so porting those solution which have been running on different Unix systems for years should not be too difficult.
- Apple needs to promote more their solution to people who want to put together a cheap and fast solution for HPC. The Xserve certainly delivers in that, why not having a version of OS X specially built for HPC (as proposed by buterbkl)? Or at least Apple should provide wide support in order that people can install their tools without any issues on OS X by maybe providing an option install on OS X install cd that contains all relevant tools.
- Maybe Apple should also consider to be more present in this market with higher end solution, Apple own built clusters solutions based on Xserve maybe?
...
i would love see three things:
1) a working spotlight - tigers was rudimental implented, in leopard its better meanwhile but the searchresults dont look reliable. and if you host large paper collections on your computer and do your research and look ups based on spotlight you need a trusted system. but somehow unfortunately it is not :( . an example:
Remote & Sensing -> 2092 results
Remote | Sensing -> 289 results
somehow i thought the results would look the exact opposite ;) so who tells me that the rest of the showed up results are reliable :/ not good at all.
2) as mentioned in previouse posts it would be nice if apple would make most of its major apps pluginable so that there is no need for dirty hacks and people are able to add proper extra functionality.
3) I think it would be nice if apple would open up a bit more regarding future developments. at the moment everything runs behind closed curtains. and even devs under nda paying much money are left in the grey about whats coming up. if the contact between apple and third party devs would be closer i think the whole community would benefit.
Re: MATLAB
This is somewhat off-topic, but I think that before tackling the X11 issue, releasing a 64bit version of MATLAB for OS X should be Mathworks' priority. If the Science Division could 'prod them along' in that direction, it would be nice.
A few suggestions
A few things that would make my life easier.
Linkback (http://www.linkbackproject.org/) or something similar, I want to be able to double click on a chemical structure embedded in a document and have it open in a Chemical structure editor.
Allow multiple spotlight plugins index the same file type, at the moment only one can, so for example I can't index new meta tags on .png file.
The iWork suite is nice but really lacks the muscle needed to really make it a "work" set of applications. Either devote more resources, or as has mentioned before, adopt a plugin framework that allows other developers to do so.
Re: Does Apple really care about HPC?
@Kevin,
As for "commonly used software" for Linux, I've rarely had issues porting if the source code is available. There are, obviously cases where there are problems -- that's why Fink and MacPorts exist. If you have particular science packages you'd like to see on the Mac, you should be speaking up on MacResearch! We can certainly help.
As far as "everything we do need," you mention Lapack, Atlas, OpenMPI... All of these are exposed in Leopard.
ls -l /usr/lib/libatlas.dylib
lrwxr-xr-x 1 root wheel 112 Oct 29 14:32 /usr/lib/libatlas.dylib@ -> ../..//System/Library/Frameworks/Accelerate.framework/Frameworks/vecLib.framework/Versions/Current/libBLAS.dylib
Now I don't have the same filesystem needs as you. I do agree (as do many people on here) that Apple needs to work on some new HPC storage options and parallel filesystems.
We're in a world of exponential data. This is as much true for Apple's traditional strengths (e.g., graphics, video, animation, movies...) as for HPC science. You'd think Pixar would want a nice high-performance, high reliability Xsan and RAID...
HPC Xserve
One of my big gripes about the Xserve as it stands, is that it doesn't seem intended for someone buying more than a few at a time. Even if you buy 20-30, you'd prefer different packaging!
Would it be so hard to have a separate HPC build, which includes one package with the manual, DVDs, rack tools, etc. (if needed) and then include only the parts I need in the rest of the boxes? I don't need X copies of the DVDs. I don't need X copies of the screwdrivers. I only want the short rails for my rack.
This seems like it would go with the "greener Apple" too, since there would be less packaging and less waste.
Maybe this seems like a small market, but to me, it seems like a few good cluster orders (e.g., even 50+ nodes) makes a healthy addition to the bottom line.
On SciTools
I think the drawback with an Apple SciTools package is that Apple will would want to provide a stable, integrated set of production tools that they release every other year or so, basically at the release rate of the Developer tools. This rate is a bit too slow - openMPI, python, numpy/scipy are moving much faster than that, and I think even the compilers in the developer tools should be updated more quickly than they are (but I don't want the bleeding edge, either.)
The MR uber package addresses some of these things, but I already get a lot from fink and the uber package would have to play nice with that, as well as the other package managment systems out there. A second issue is that the MR audience is quite broad; how will you decide what to include and what to leave out? If I only want a small number of the programs out of the bunch, I'll just get them another way. But if the MR package didn't go in to field-specific software and was compatible with other package managers, it could be quite valuable.
For me, fink works pretty well; the only stumbling block is that I am at the mercy of package maintainers for being up-to-date. I deeply appreciate all the unpaid work those maintainers do, but I think if Apple or MR (or anyone, actually) could throw more dedicated support at the most popular scientific/mathematic fink packages it would have a big impact.
V. Gregory Weirs
Sandia National Laboratories
Re: Sci Tools
Aside from the fact that I think Apple should add official support for gfortran (and openmpi/gfortran bindings) in Leopard, I agree with the previous poster that macresearch.org's efforts would be best spent improving scientific package support in fink/macports, rather than creating our own sci tools distribution.
Working with fink/macports has the benefits of building on a large existing base of tools, and things like gcc/gfortran are already actively maintained. I think those projects just need help keeping niche scientific packages up to date. (scipy, texlive instead of tetex, py-pyx, octave, etc.) Plus, macports has a _very_ nice bug tracker which can be used to keep track of problems and requests.
Re: Does Apple really care about HPC?
@ghutchis,
Thank you for your reply. I did not mean to imply that Lapack, Atlas, etc. were not in Leopard, simply that there needs to be an HPC version of MacOS X with all of these things already there and tuned and, just as importantly, all the things that an HPC environment doesn't need (eg. mail server, iCal server, etc.) stripped out.
We're currently in the process of setting up a test cluster in preparation for moving from a 32-bit Linux cluster to a 64-bit Linux cluster. Me and another guy here have spent a lot of hours lately building all of the aforementioned libraries (and a few others) for 64-bit and then testing them. If Apple had an HPC version of MacOS X that "just worked" (like so many other Apple products do) out of the box ... man, could I go for something like that right about now!
I'm in total agreement with you in regards to your comments regarding Xserve RAID, Xsan, and parallel filesystems.
However, even if Apple (possibly in conjunction with MacResearch) granted me all of the above, without the ability to run OSG jobs (and specifically CMS jobs via OSG) on a Mac cluster, there's no chance of us considering a Mac cluster. Again, I find that a shame, as there are a couple of groups here that have small Mac clusters for specific purposes and they all absolutely love the performance of the Intel-based Xserves.
Thanks for taking the time to reply to my original comments...
Kevin
Re: Sci Tools
Of course Fink and MacPorts are great efforts, and I don't want to take anything away from their contribution at all. But I still miss something, and I have trouble putting my finger on. For example, even though fink and macports have gfortran, I will nearly always go to Gaurav's hpc.sourceforge.net web site to download a package installer, or even build it myself using the old configure/make/make install approach, rather than using one of the package managers.
Why? I'm not really sure. I guess the barrier is just that little bit more. I like the cleanness and ease of an installer, or drag-and-drop install. If there were a package or drag-and-drop install with 500MB of standard scientific libraries and tools, even if it was ultimately built using fink and macports, I would probably install it just for the ease of use and knowledge that it will work out of the box.
I guess I have had enough disappointment with modules not installing properly via the package managers, that I have lost a bit of faith in them. I would rather have something that just worked, the way it is when you download an application and install that.
Drew
---------------------------
Drew McCormack
http://www.maccoremac.com
http://www.macanics.net
http://www.macresearch.org
Make porting of Unix/Linux software easier
by using the same names/paths for common libraries, h-files etc.
My 2 cents.
If I had 5 minutes with Apple's customers...
I was thinking about this last night ... we, as Apple customers, are partly to blame for things like the Xserve RAID. Think about it; it's 2008 and Apple is still selling an array without SATA drives or redundant controllers! How can they get away with that? Simple; at least some of us are still *buying* them!
I wonder how quickly Apple would introduce a new Xserve RAID if we suddenly stopped buying the current model? Think about the Cinema Displays; I can buy a 20" Apple Cinema Display for $599 or I can buy a Samsung 22" LCD monitor with better technical specs for a little over $300. How quickly would Apple introduce new Cinema Displays (or make appropriate price adjustments on the current ones) if we would all stop buying them just because they've got the Apple logo on them?
Now please don't get me wrong, I'm not trying to bash Apple here. I like Apple. I love most of their products and think that they are a good value for the money. But there's some things like the two that I just mentioned that shouldn't even be happening.
To a certain extent, Apple is going to go in the direction Uncle Steve wants it to go - period. But there are certain areas where we can nudge Apple in the direction we want them to go. I would encourage all of us to do more than just post suggestions here and hope that someone: 1) actually gets to talk with somebody at Apple about this, and 2) that anything actually comes from said meeting if it occurs. I would encourage all of us to talk to Apple with the loudest, clearest voice any of us have: our money.
Thanks for your consideration...
Kevin
Re: GPU computing using CUDA -- Yes! I completely second that.
All is in the title. I was asking Apple several times about CUDA support and just didn't get a reply. As I was working on a research project that I had funding for, I had to switch to a non-Apple solution, quite against my liking. I also find it technically very annoying that technology available for Linux is not ported to the BSD/Darwin/Apple platform. IMHO it shouldn't have been that much effort once you have a working UNIX-compliant implementation.
core system
Apple should just quit sticking their hands into things that function well and are well designed to start with. The threading queue for Apache is the thing that kills me. But hacking that trace utility to hide iTunes. What a waste of everybody's time. And isn't that why we use Apple?
A good start to improvement is not wasting productive time. Ours or theirs. Forget adding things.
Apple's (OS X) share on the science markets?
Thanks very much for the smart contributions. I've learnt a lot and trying to compile my questions.
A general question: do we know approximately how many scientists, labs, departments, universities, research institutes are using Macs? Have you heard on any good survey?
Is the science or the medical market share is different compared to the general? Or the ratio is higher? And compared to the technology sphere?
Market Share
I don't have any figures, but I would guess Apple has a slightly higher share of the science market than their overall market share. Macs are still used a lot in education, including universities. My group (around 30) is all Mac, for example (I'm in a Chemistry department).
There was a study recently showing that around 40% of first year students at Stanford (I think) had a Mac. Apparently they are catching on with the new students, so hopefully that trend will continue.
Drew
---------------------------
Drew McCormack
http://www.maccoremac.com
http://www.macanics.net
http://www.macresearch.org
Some things I miss
- 64bit version of Matlab (hell, there's a 64bit linux version so it can't be that hard) and in general 64bit support of all major science apps but thats not up to Apple to be honest.
- beter cross platform support or linux/unix compatibility (as most science number crunching is either done on linux or a unix flavor portability should be enhanced.
Follow up?
Did this meeting ever occur? Did anything ever come out of it?
V. Gregory Weirs
Re: Follow up?
I don't think it has happened yet, but it is planned.
I believe it will appear at this URL (http://pimm.wordpress.com/) when it happens. We'll link to it at MR too.
Drew
---------------------------
Drew McCormack
http://www.maccoremac.com
http://www.macanics.net
http://www.macresearch.org
Pages 2.0 and Papers 1.7
Pages 2.0 with its improvements is a great alternative to word and I hope we get some kind of EndNote support soon. One other support tool to assist Scientists is Papers 1.7 (http://mekentosj.com/papers/)
-----------------------
http://www.amandeep.org/