Advanced Fortran 90: Callbacks with the Transfer Function
Author: Drew McCormack
Website: http://www.macanics.net
This post is not specific to the Mac, but there are probably many MR readers that have to use Fortran in some aspect of their work, so it certainly can’t hurt…
Callbacks
A few weeks ago I was lamenting the difficulty of implementing decent callback functionality in Fortran 90. A callback is common place in C programming; it basically allows you to customize the behavior of a subroutine. Take this simple example:
#include <stdio.h>
void Woof()
{
printf("Woof\n");
}
void Meouw()
{
printf("Meouw\n");
}
typedef void (*SoundFunction)();
void MakeSoundTenTimes(SoundFunction soundFunc)
{
int i;
for ( i = 0; i < 10; ++i ) soundFunc();
}
int main()
{
MakeSoundTenTimes(Woof);
MakeSoundTenTimes(Meouw);
}
The point of this rather obscure example is that the main program can call the MakeSoundTenTimes function passing a function pointer. (If you are not familiar with C’s cryptic function pointer syntax, don’t worry about it: it is not important.) The loop in MakeSoundTenTimes can call the function passed in, effectively customizing its behavior. It’s a bit like rewriting the inside of the MakeSoundTenTimes each time: the function passed determines the results.
You can actually do this in Fortran 90. It looks like this:
module Sounds
contains
subroutine Woof()
print *,'Woof'
end subroutine
subroutine Meouw()
print *,'Meouw'
end subroutine
subroutine MakeSoundTenTimes(soundFunc)
integer :: i
interface
subroutine soundFunc()
end subroutine
end interface
do i = 1, 10
call soundFunc()
enddo
end subroutine
end module
program main
use Sounds
call MakeSoundTenTimes(Woof)
call MakeSoundTenTimes(Meouw)
end program
Callbacks with Arbitrary Arguments
This works fine, but problems start to arise if you need to pass some data to the callback. In C, it is conventional to include a void pointer, which can point to any type of data. But Fortran 90 doesn’t have void pointers, so how do you pass arbitrary data to the callback? To see how you can do this, let’s again begin with an example from C which passes data via a void pointer:
#include <stdio.h>
void IncrementAndPrintFloat(void *data)
{
double *d = data;
(*d)++;
printf("%f\n", *d);
}
void IncrementAndPrintInteger(void *data)
{
int *i = data;
(*i)++;
printf("%d\n", *i);
}
typedef void (*IncrementFunction)(void*);
void IncrementTenTimes(IncrementFunction incrFunc, void *data)
{
int i;
for ( i = 0; i < 10; ++i ) incrFunc(data);
}
int main()
{
double f = 5.0;
int i = 10;
IncrementTenTimes(IncrementAndPrintFloat, &f);
IncrementTenTimes(IncrementAndPrintInteger, &i);
}
This is quite similar to the original example, but now the callbacks take a void* argument, as does the IncrementTenTimes function that calls back. The callback functions know what type of data is passed to them, so they cast the void pointer to the appropriate type, increment, and print. The IncrementTenTimes function, on the other hand, does not know anything about what the data represents. It is just a generic pointer that gets passed on to a callback function. This is actually a strength of the callback pattern — the behavior of the calling-back function and the callback function are largely decoupled, facilitating code reuse.
For a long time, I didn’t think you could have this sort of callback in Fortran 90, but I recently realized that you can. It involves a relatively obscure intrinsic function called transfer. If you ask the majority of Fortran programmers what transfer does, chances are you will get a blank look or a rather vague answer. What transfer actually is is a means of casting data from one type to another.
So what would the above C example look like in Fortran 90 with the transfer function. Here it is:
module Increments
contains
subroutine IncrementAndPrintReal(data)
character(len=1) :: data(:)
real :: r
r = transfer(data, r)
r = r + 1.0
print *,r
data = transfer(r, data)
end subroutine
subroutine IncrementAndPrintInteger(data)
character(len=1) :: data(:)
integer :: i
i = transfer(data, i)
i = i + 1
print *,i
data = transfer(i, data)
end subroutine
subroutine IncrementTenTimes(incrFunc, data)
character(len=1) :: data(:)
integer :: i
interface
subroutine incrFunc(data)
character(len=1) :: data(:)
end subroutine
end interface
do i = 1, 10
call incrFunc(data)
enddo
end subroutine
end module
program main
use Increments
character(len=1), allocatable :: data(:)
integer :: lengthData
real :: r = 5.0
integer :: i = 10
lengthData = size(transfer(r, data))
allocate(data(lengthData))
data = transfer(r, data)
call IncrementTenTimes(IncrementAndPrintReal, data)
deallocate(data)
lengthData = size(transfer(i, data))
allocate(data(lengthData))
data = transfer(i, data)
call IncrementTenTimes(IncrementAndPrintInteger, data)
end program
Now I’m the first to admit this is a little more verbose than the C version, and that’s putting it mildly, but it is possible to do, and could be a very handy tool in certain instances.
So how does it work? Let’s take a transfer function call from the main program, and dissect that:
data = transfer(r, data)
What this does is copy the bytes of the variable r, returning it with the type of data. The second argument to the transfer function is there purely to tell the function what type of data it should return. The data returned by transfer is then stored in the variable data, which is an array of characters.
If the second argument to transfer is an array, no matter how long, it will figure out how big the return array needs to be to fully accommodate the type passed in as first argument. That’s pretty nifty, and you can see that we use that to determine how big our data array needs to be to be able to contain the real or integer variable being passed. In particular, this line
lengthData = size(transfer(r, data))
does the conversion, but only uses the result to send to the size intrinsic to determine how long the data array should be. The data array is then allocated (malloced for our C friends) and transfer is called again, this time with the result actually getting stored in data.
One final thing before signing off: we have used an array of character(len=1) to store generic data here, but you can use any array type you like. transfer will figure out from the type you are using how many elements the array needs to have. I have chosen to use an array of characters, because a character usually corresponds to one byte, and that way I won’t be wasting any memory. But you could also use an array of integers, for example, and then your data granularity would be something like 4-bytes, depending on the operating system and compiler.
Fortran is often underestimated as a programming language, but there is plenty in there for scientific developers. Some aspects of the language, like transfer, can seem a bit abstract to begin with, and their potential not fully realized. Hopefully this little tutorial has shown you that transfer is useful, and encourages you to seek out other uses yourself. (Hint: You can also build data containers for generic types, like dynamic arrays and dictionaries, using transfer.)



Comments
Uses for transfer
First off, thanks for this series of articles Drew, they are very interesting.
Transfer is an interesting and useful function. I have used it to rewrite a C hashing function in fortran 90 (the original is here http://burtleburtle.net/bob/hash/evahash.html). The Fortran 90 version cannot store arbitrary types, an assignment routine has to be written for each type to be stored and retrieved. This isn't a huge problem, but it does stop me from storing references to any old thing and so limits it's generality.
I use this primarily for command line argument processing, where I can use generic routines for returning a command line argument as a user-defined type and then utilise user-defined assignment to coerce my command line argument to a real/integer/character as defined in the code. Really quite useful.
Having used perl fairly extensively I got used to the convenience of associative arrays (hashes) and arrays (flexible, growable arrays of real, integer, character). So I also wrote a module for variable size arrays, so I can push/pop/splice elements on to a simple array of real/integer/varying string. These little work arrays are very useful and make it very convenient to write code.
Thanks
Hi Aidan,
Thanks for the comment. Using transfer to create a hash is a great idea. I might give that one a try myself.
Your comment shows that there are people out there using this stuff, but often it isn't discussed in public forums. That's why I thought it would be useful to write about transfer. Otherwise, people are left to their own devices to figure it all out themselves.
Thanks again.
Drew
---------------------------
Drew McCormack
http://www.macanics.net
http://www.macresearch.org
Thanks Drew + Derived Types and "transfer
Drew, thanks for the very informative article.
Just for fun, I wanted to see if one could pack a derived type into the transfer function and pass it using the callback. I was somewhat amazed that it works.
Example: I added the following code to your example, a derived type in the module, a new printing function, and a new calling function
New Type
TYPE structure
real s
integer q
real, allocatable :: flt1d(:)
END TYPE structure
New subroutine in the module
subroutine IncrementAndPrintStructure(data)
character(len=1) :: data(:)
type(structure) :: t0
t0 = transfer(data, t0)
print *, t0%flt1d
data = transfer(t0, data)
end subroutine
New code in the main program
lengthData = size(transfer(t, data))
allocate(data(lengthData))
data = transfer(t, data)
call IncrementTenTimes(IncrementAndPrintStructure, data)
deallocate(data)
------
This works - which is pretty interesting. Since it doubles the memory for a large application, it may not be the best way to pass large structured data sets around generically, but it might prove to be very useful.
This was tested on a Mac Pro using both the intel fortran compiler and g95.
Thanks again.
Lou Wicker
More use for transfer
If you want to see the hash code I'll let it out in all it's ugliness. It was remarkably easy to rewrite the C stuff, one of my quicker coding efforts.
There does seem to be a culture of "rolling your own" amongst Fortran coders. This probably reflects the nature of the problems Fortran users tackle (specialist science in the main) and the limitations of the language. The new features of fortran 90/95/2003 make it much easier to write portable, reusable code. I'd like to see cooperatively developed Fortran modules that do all those useful little jobs that we all need to do.
I guess the f2kcli and iso_varying_string modules are examples of this (not that I use the former).
Does MR want to host/spruik some community type modules?
Hash
I'd love to see your hash function. I tried my own the other day after you posted. It was pretty easy, but maybe you have a better one. If you show me yours, I'll show you mine ;-)
We have been talking for some time at MR about having some sort of project hosting or source code repository. Don't know the current status. Thanks for the input though.
Drew
---------------------------
Drew McCormack
http://www.macanics.net
http://www.macresearch.org
Re: Hash
"We have been talking for some time at MR about having some sort of project hosting or source code repository. Don't know the current status. Thanks for the input though."
It's still on the books as something to do. Unfortunately, things seem to be taking longer than we've anticipated for a number of site changes. Hopefully sometime soon in the new year, we'll have a lot of new enhancements to Macresearch to make the site even more useful.
In the meantime if you have some code or a project that you'd like hosted (either as pure code, or even as part of a tutorial), let us (or specifically Drew) know and we'd be happen to make arrangements to house it on the server. All credit would go to you of course.
Thanks for the interest and support!
Dave
Limitations on transfer
Yeah, transfer works with just about anything, included user-defined types. If the compiler knows how big something is, it can pack it into a data buffer.
If you are worried about the size of the data passed, here's a tip: pass a user-defined type containing a pointer instead. Take this:
Then you just pass the pointer via transfer:
And in sub:
The reason you need to make a user-defined type for the pointer is that if you don't, it's not clear whether the transfer function should serialize the pointer passed, or the thing that it points to. With a user-defined type, it is clear.
There is only one thing that I tried that didn't work with transfer: functions and subroutines. I thought it would cool if you could try to store a subroutine argument as data; this would allow some very powerful programming, as seen in C where you can have function pointers. But it didn't work. At least, I couldn't get it to work. Guess we have to wait for Fortran 2003 for that one...
Drew
---------------------------
Drew McCormack
http://www.macanics.net
http://www.macresearch.org
MPI and transfer
Nice articles!
Just info though on another limitation. I had hoped to use the above ideas to store derived types as character arrays, and then send them across processors in a parallel MPI environment. This does actually work very nicely, but I have read that it isn't recommended. As long as all nodes have the same arhcitecture, everything is fine. But if they don't, it ain't! (e.g. if one node is big-endian, and the other little endian, the data gets mangled). Shame, as transfer would be a very nice solution to handling derived types in parallel coding.
Dave
thanks!
very informative
power of c in f
thanks!