Advanced Fortran 90: Callbacks with the Transfer Function

Author: Drew McCormack
Website: http://www.mentalfaculty.com

This post is not specific to the Mac, but there are probably many MR readers that have to use Fortran in some aspect of their work, so it certainly can’t hurt…

Callbacks

A few weeks ago I was lamenting the difficulty of implementing decent callback functionality in Fortran 90. A callback is common place in C programming; it basically allows you to customize the behavior of a subroutine. Take this simple example:

#include <stdio.h>

void Woof()  
{
    printf("Woof\n");
}

void Meouw() 
{
    printf("Meouw\n");
}

typedef void (*SoundFunction)();
void MakeSoundTenTimes(SoundFunction soundFunc)
{
    int i;  
    for ( i = 0; i < 10; ++i ) soundFunc();
}

int main()  
{
    MakeSoundTenTimes(Woof);
    MakeSoundTenTimes(Meouw);
}

The point of this rather obscure example is that the main program can call the MakeSoundTenTimes function passing a function pointer. (If you are not familiar with C’s cryptic function pointer syntax, don’t worry about it: it is not important.) The loop in MakeSoundTenTimes can call the function passed in, effectively customizing its behavior. It’s a bit like rewriting the inside of the MakeSoundTenTimes each time: the function passed determines the results.

You can actually do this in Fortran 90. It looks like this:

module Sounds

contains

  subroutine Woof()
    print *,'Woof'
  end subroutine

  subroutine Meouw() 
    print *,'Meouw'
  end subroutine

  subroutine MakeSoundTenTimes(soundFunc)
    integer :: i
    interface
      subroutine soundFunc()
      end subroutine
    end interface
    do i = 1, 10 
      call soundFunc()
    enddo
  end subroutine

end module

program main
  use Sounds
  call MakeSoundTenTimes(Woof)
  call MakeSoundTenTimes(Meouw)
end program

Callbacks with Arbitrary Arguments

This works fine, but problems start to arise if you need to pass some data to the callback. In C, it is conventional to include a void pointer, which can point to any type of data. But Fortran 90 doesn’t have void pointers, so how do you pass arbitrary data to the callback? To see how you can do this, let’s again begin with an example from C which passes data via a void pointer:

#include <stdio.h>

void IncrementAndPrintFloat(void *data)  
{
    double *d = data; 
    (*d)++; 
    printf("%f\n", *d);
}

void IncrementAndPrintInteger(void *data)  
{
    int *i = data; 
    (*i)++; 
    printf("%d\n", *i);
}

typedef void (*IncrementFunction)(void*);
void IncrementTenTimes(IncrementFunction incrFunc, void *data)
{
    int i;  
    for ( i = 0; i < 10; ++i ) incrFunc(data);
}

int main()  
{
    double f = 5.0;
    int i = 10; 
    IncrementTenTimes(IncrementAndPrintFloat, &f);
    IncrementTenTimes(IncrementAndPrintInteger, &i);
}

This is quite similar to the original example, but now the callbacks take a void* argument, as does the IncrementTenTimes function that calls back. The callback functions know what type of data is passed to them, so they cast the void pointer to the appropriate type, increment, and print. The IncrementTenTimes function, on the other hand, does not know anything about what the data represents. It is just a generic pointer that gets passed on to a callback function. This is actually a strength of the callback pattern — the behavior of the calling-back function and the callback function are largely decoupled, facilitating code reuse.

For a long time, I didn’t think you could have this sort of callback in Fortran 90, but I recently realized that you can. It involves a relatively obscure intrinsic function called transfer. If you ask the majority of Fortran programmers what transfer does, chances are you will get a blank look or a rather vague answer. What transfer actually is is a means of casting data from one type to another.

So what would the above C example look like in Fortran 90 with the transfer function. Here it is:

module Increments

contains

  subroutine IncrementAndPrintReal(data)
    character(len=1) :: data(:)
    real             :: r
    r = transfer(data, r)
    r = r + 1.0 
    print *,r
    data = transfer(r, data)
  end subroutine

  subroutine IncrementAndPrintInteger(data)
    character(len=1) :: data(:) 
    integer          :: i    
    i = transfer(data, i)
    i = i + 1 
    print *,i
    data = transfer(i, data)
  end subroutine

  subroutine IncrementTenTimes(incrFunc, data)
    character(len=1) :: data(:) 
    integer :: i
    interface
      subroutine incrFunc(data)
        character(len=1) :: data(:) 
      end subroutine
    end interface
    do i = 1, 10 
      call incrFunc(data)
    enddo   
  end subroutine

end module

program main
  use Increments
  character(len=1), allocatable :: data(:) 
  integer                       :: lengthData
  real                          :: r = 5.0 
  integer                       :: i = 10

  lengthData = size(transfer(r, data))
  allocate(data(lengthData))
  data = transfer(r, data)
  call IncrementTenTimes(IncrementAndPrintReal, data)
  deallocate(data)

  lengthData = size(transfer(i, data))
  allocate(data(lengthData))
  data = transfer(i, data)
  call IncrementTenTimes(IncrementAndPrintInteger, data)

end program

Now I’m the first to admit this is a little more verbose than the C version, and that’s putting it mildly, but it is possible to do, and could be a very handy tool in certain instances.

So how does it work? Let’s take a transfer function call from the main program, and dissect that:

data = transfer(r, data)

What this does is copy the bytes of the variable r, returning it with the type of data. The second argument to the transfer function is there purely to tell the function what type of data it should return. The data returned by transfer is then stored in the variable data, which is an array of characters.

If the second argument to transfer is an array, no matter how long, it will figure out how big the return array needs to be to fully accommodate the type passed in as first argument. That’s pretty nifty, and you can see that we use that to determine how big our data array needs to be to be able to contain the real or integer variable being passed. In particular, this line

lengthData = size(transfer(r, data))

does the conversion, but only uses the result to send to the size intrinsic to determine how long the data array should be. The data array is then allocated (malloced for our C friends) and transfer is called again, this time with the result actually getting stored in data.

One final thing before signing off: we have used an array of character(len=1) to store generic data here, but you can use any array type you like. transfer will figure out from the type you are using how many elements the array needs to have. I have chosen to use an array of characters, because a character usually corresponds to one byte, and that way I won’t be wasting any memory. But you could also use an array of integers, for example, and then your data granularity would be something like 4-bytes, depending on the operating system and compiler.

Fortran is often underestimated as a programming language, but there is plenty in there for scientific developers. Some aspects of the language, like transfer, can seem a bit abstract to begin with, and their potential not fully realized. Hopefully this little tutorial has shown you that transfer is useful, and encourages you to seek out other uses yourself. (Hint: You can also build data containers for generic types, like dynamic arrays and dictionaries, using transfer.)

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Uses for transfer

First off, thanks for this series of articles Drew, they are very interesting.

Transfer is an interesting and useful function. I have used it to rewrite a C hashing function in fortran 90 (the original is here http://burtleburtle.net/bob/hash/evahash.html). The Fortran 90 version cannot store arbitrary types, an assignment routine has to be written for each type to be stored and retrieved. This isn't a huge problem, but it does stop me from storing references to any old thing and so limits it's generality.

I use this primarily for command line argument processing, where I can use generic routines for returning a command line argument as a user-defined type and then utilise user-defined assignment to coerce my command line argument to a real/integer/character as defined in the code. Really quite useful.

Having used perl fairly extensively I got used to the convenience of associative arrays (hashes) and arrays (flexible, growable arrays of real, integer, character). So I also wrote a module for variable size arrays, so I can push/pop/splice elements on to a simple array of real/integer/varying string. These little work arrays are very useful and make it very convenient to write code.

Thanks

Hi Aidan,
Thanks for the comment. Using transfer to create a hash is a great idea. I might give that one a try myself.
Your comment shows that there are people out there using this stuff, but often it isn't discussed in public forums. That's why I thought it would be useful to write about transfer. Otherwise, people are left to their own devices to figure it all out themselves.
Thanks again.
Drew

---------------------------
Drew McCormack
http://www.macanics.net
http://www.macresearch.org

Thanks Drew + Derived Types and "transfer

Drew, thanks for the very informative article.

Just for fun, I wanted to see if one could pack a derived type into the transfer function and pass it using the callback. I was somewhat amazed that it works.

Example: I added the following code to your example, a derived type in the module, a new printing function, and a new calling function

New Type

TYPE structure
real s
integer q
real, allocatable :: flt1d(:)
END TYPE structure

New subroutine in the module

subroutine IncrementAndPrintStructure(data)
character(len=1) :: data(:)
type(structure) :: t0
t0 = transfer(data, t0)
print *, t0%flt1d
data = transfer(t0, data)
end subroutine

New code in the main program

lengthData = size(transfer(t, data))
allocate(data(lengthData))
data = transfer(t, data)
call IncrementTenTimes(IncrementAndPrintStructure, data)
deallocate(data)

------
This works - which is pretty interesting. Since it doubles the memory for a large application, it may not be the best way to pass large structured data sets around generically, but it might prove to be very useful.

This was tested on a Mac Pro using both the intel fortran compiler and g95.

Thanks again.

Lou Wicker

More use for transfer

If you want to see the hash code I'll let it out in all it's ugliness. It was remarkably easy to rewrite the C stuff, one of my quicker coding efforts.

There does seem to be a culture of "rolling your own" amongst Fortran coders. This probably reflects the nature of the problems Fortran users tackle (specialist science in the main) and the limitations of the language. The new features of fortran 90/95/2003 make it much easier to write portable, reusable code. I'd like to see cooperatively developed Fortran modules that do all those useful little jobs that we all need to do.

I guess the f2kcli and iso_varying_string modules are examples of this (not that I use the former).

Does MR want to host/spruik some community type modules?

Hash

I'd love to see your hash function. I tried my own the other day after you posted. It was pretty easy, but maybe you have a better one. If you show me yours, I'll show you mine ;-)

We have been talking for some time at MR about having some sort of project hosting or source code repository. Don't know the current status. Thanks for the input though.

Drew

---------------------------
Drew McCormack
http://www.macanics.net
http://www.macresearch.org

Re: Hash

"We have been talking for some time at MR about having some sort of project hosting or source code repository. Don't know the current status. Thanks for the input though."

It's still on the books as something to do. Unfortunately, things seem to be taking longer than we've anticipated for a number of site changes. Hopefully sometime soon in the new year, we'll have a lot of new enhancements to Macresearch to make the site even more useful.

In the meantime if you have some code or a project that you'd like hosted (either as pure code, or even as part of a tutorial), let us (or specifically Drew) know and we'd be happen to make arrangements to house it on the server. All credit would go to you of course.

Thanks for the interest and support!

Dave

Limitations on transfer

Yeah, transfer works with just about anything, included user-defined types. If the compiler knows how big something is, it can pack it into a data buffer.

If you are worried about the size of the data passed, here's a tip: pass a user-defined type containing a pointer instead. Take this:

type BigData
  real :: d(1000)
end type

type BigDataPointer
  type(BigData), pointer :: p
end type

Then you just pass the pointer via transfer:

type (BigData), target :: bd
type (BigDataPointer) :: bdp
character(len=1)        :: byte(1)
bd%d = 10.0
bdp%p => bd
call sub( transfer( bdp, byte) )

And in sub:

subroutine sub( d )
character(len=1) :: d(:)
type (BigDataPointer) :: bdp
bdp = transfer(d, bdp)
print *, bdp%p  
end subroutine

The reason you need to make a user-defined type for the pointer is that if you don't, it's not clear whether the transfer function should serialize the pointer passed, or the thing that it points to. With a user-defined type, it is clear.

There is only one thing that I tried that didn't work with transfer: functions and subroutines. I thought it would cool if you could try to store a subroutine argument as data; this would allow some very powerful programming, as seen in C where you can have function pointers. But it didn't work. At least, I couldn't get it to work. Guess we have to wait for Fortran 2003 for that one...

Drew

---------------------------
Drew McCormack
http://www.macanics.net
http://www.macresearch.org

MPI and transfer

Nice articles!

Just info though on another limitation. I had hoped to use the above ideas to store derived types as character arrays, and then send them across processors in a parallel MPI environment. This does actually work very nicely, but I have read that it isn't recommended. As long as all nodes have the same arhcitecture, everything is fine. But if they don't, it ain't! (e.g. if one node is big-endian, and the other little endian, the data gets mangled). Shame, as transfer would be a very nice solution to handling derived types in parallel coding.

Dave

thanks!

very informative
power of c in f
thanks!

We should thank you for

We should thank you for giving such a wonderful blog. Your site happens to be not only informative but also very imaginative too. We find a limited number of experts who can think to write technical articles that creatively. All of us are on the lookout for information on something like this. I Myself have gone through several blogs to build up on knowledge about this.We look forward to the next posts !!

clorox bath wand

brown betty teapot

Very useful article, but now

Very useful article, but now I have a related question. I came across this article searching for something else (but related) on the net. Perhaps that can be done with 'transfer' as well, but I don't see how. Basically, I have a subroutine (say) func_1, which is interfaced, so it can accept a few data types:


module my_module

interface func_1
module procedure func_1_r8
module procedure func_1_i4
...
end interface

subroutine func_1_r8(x)
real(8), intent(in) :: x
...
end subroutine func_1_r8

subroutine func_1_i4(x)
integer(4), intent(in) :: x
...
end subroutine func_1_i4

end module my_module

Now I want to call this from func_2, and pass a value from func_2 to func_1, which in turn should be an input argument to func_2. Something like this:


subroutine func_2(y,x)
use my_module
...
call func_1(x)
...
end subroutine func_2

where x can be a real(8) or an integer(4). The trouble is obvious; though func_1 can handle both types, I need to declare x within func_2, for which I need to interface func_2 as well, which seems rather needless. Is there a way, perhaps using transfer, that I can use to pass x through func_2 to func_1, without func_2 caring about the type or shape of x?