These are chat archives for Fortran-FOSS-Programmers/General-Discussion

21st
Apr 2017
Stefano Zaghi
@szaghi
Apr 21 2017 08:49

Dear @/all ,
I am facing on a very big issue with GNU gfortran and polymorphic functions (for your interest see this) related to serious memory leaks generation. If the bug is confirmed I think the fix will be difficult and it will require a lot of time.

I am wondering if there are some techniques or approaches (maybe developed when f2003/2008 features were not yet widely available) to mimic polymorphism. I would like to save the work done in last months and reuse as much as possible all the polymorphic libraries. I found a paper describing how to achieve multi-inheritance in f2003, maybe there are similar papers on how to achieve a sort of polymorphism without class(...), allocatable...

Thank you in advance for any hints!

Cheers

Chris MacMackin
@cmacmackin
Apr 21 2017 11:17
There is already a report of this bug: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60913. My way of dealing with it (which doesn't stop all memory leaks but is enough to keep them to a manageable level) is that described on pages 236-244 of Rouson, Xia, and Xu's Scientific Software Design: The Object-Oriented Way. The main disadvantage of this approach is it means your routines can no longer be pure. You can see an example of this approach in my factual library. Let me know if you want me to explain some of the details of it.
Stefano Zaghi
@szaghi
Apr 21 2017 12:41
Hi Chris! @cmacmackin Thank you very much for the hints. I searched for similar issues, but I missed it yesterday. Surprising, now that you point it out, I am quite sure I have already it (probably again from one of your hints). This makes me very sad: that report is dated 2014! I am also surprise about the comments of Damian in that report (@rouson). I am also surprised that in his book there is a workaround, I'll go to re-study it right now. Frankly speaking, this is a nightmare (see also the comments of Ian Harvey): I am seriously thinking to left Fortran...
Stefano Zaghi
@szaghi
Apr 21 2017 13:35
@cmacmackin @rouson I just re-read the note Warning: Calculus May Cause Memory Loss, Bloating and Insomnia on page 238... arghhhhh tell me this is a nightmare!!!!
Stefano Zaghi
@szaghi
Apr 21 2017 15:26

@cmacmackin Chris, I need your help...

I re-read the @rouson book, but that part looks obscure in the description of the leak-free hermetic field (this is probably why I missed in my first lecture...). I have then studied your factual but I have difficult to understand the *rationale".

I am focused on how you implement the operators, say this multiplication that I report here

  function array_scalar_sf_m_sf(this,rhs) result(res)
    class(array_scalar_field), intent(in) :: this
    class(scalar_field), intent(in) :: rhs
    class(scalar_field), allocatable :: res !! The restult of this operation
    class(array_scalar_field), allocatable :: local

    call this%guard_temp(); call rhs%guard_temp() 
   allocate(local, mold=this)
    call local%assign_meta_data(this)
    select type(rhs)
    class is(array_scalar_field)
      local%field_data%array = this%field_data%array * rhs%field_data%array
    class is(uniform_scalar_field)
      local%field_data%array = this%field_data%array * rhs%get_value()
    end select
    call move_alloc(local,res)
    call res%set_temp()
    call this%clean_temp(); call rhs%clean_temp()
  end function array_scalar_sf_m_sf

I see that guard_temp method increase the temporary level if they are already set as temporary, so this and rhs are optional now more temporary... why this is necessary? My first guess is that this and rhs should not be a problem, but I could underestimate the function recursive execution scope I think.

Why the local is necessary? Is there something implied in the move_alloc?

Why res is set as temporary? I guess because then it can be properly finalized, but I cannot figured out who then is able to finalize res.

Chris MacMackin
@cmacmackin
Apr 21 2017 16:54
To see why this is necessary, consider the following code.
program example
  use factual_mod
  implicit none
  type(cheb1d_scalar_field) :: a, b, c

  a = b * c
  a = do_thing(b, c)
  a = do_thing(b*b, c)

contains

  function do_thing(thing1, thing2)
    class(scalar_field), intent(in) :: thing1, thing2
    class(scalar_field), allocatable :: do_thing
    call thing1%guard_temp(); call thing2%guard_temp()
    call thing1%allocate_scalar_field(do_thing)
    do_thing = thing1*thing2
    call thing1%clean_temp(); call thing2%clean_temp()
    call do_thing%set_temp()
  end function do_thing

end program example
When a = b*c is executed, b, and c are both non-temporary variables. As such, they should not be finalised. So far, so obvious. However, the actual arguments to function do_thing may or may not be temporary. If they are not temporary (e.g. when executing a = do_thing(b, c)), then this function will not finalise them and nor will the multiplication routine.
Chris MacMackin
@cmacmackin
Apr 21 2017 17:02
Memory leaks occur when the variable returned from a function are not finalised after assignment or use as an argument. (Note that this issue can sometimes be avoided by returning pointer variables and using pointer assignment, at the cost of lots of manual memory-management. ) As such, wee need to have the clean_temp method which is called at the end of all procedures using fields as arguments. However, we don't want to finalise any fields which have been assigned to variables, so we must distinguish between these ones and temporary ones which are just function results.
Chris MacMackin
@cmacmackin
Apr 21 2017 17:12

Function results may or may not be assigned to a variable, so we must assume that they are temporary, hence why set_temp is called for the function results. In the defined assignment operator, the LHS argument will be set not to be temporary so that it will not be finalised prematurely.

Where things get complicated is when a function receives a temporary field as an argument and must pass it to another procedure. This happens in do_thing when it is called in the a = do_thing(b*b, c) statement. As b*b is temporary, it will be guarded at the start of do_thing. When we pass it to the multiplication method, we don't want it to be finalised yet (as we might want to do something else with it in do_thing). As such, the call to set_temp in the multiplication routine must increment the temporary-count of this argument. That way, it will know not to finalise it when clean_temp is called in the multiplication routine. Instead, clean_temp will just decrement the temporary-count again. When the clean_temp call is made in do_thing, the count will be back down to its initial value and clean_temp will know that it can safely finalise the argument.

I hope that clarifies things.
Stefano Zaghi
@szaghi
Apr 21 2017 19:15

@cmacmackin Chris all are more clear, but indeed, I think this does not work my bug. Now I understand the care you paid on finalization, but the point is that in my test case the finalization is totally not useful... my type prototype (your fields) is something like


type :: field
   real :: i_am_static(length)
   contains
     procedure...
endtyep field

Now if I add a finalizer to field it will have no effect on the i_am_static member of the type. My leaks originate to the fact that gfortran is not able to free the static memory of class(...), allocatable function results. If I made the static member allocatable the leaks seem to vanish (but not if the allocatables are other types...) as well as if I trim out the polymorphism defining the result as type(...), allocatable the finalization is automatically done right with both static and dynamic members. So, you workaround could be very useful to fix related to dynamic members that are other class/types, but it has no effect on the static member. Tomorrow I'll try your workaround in more details, but I am going to sleep very sadly...

Anyhow, thank you very much, you are great!

Chris MacMackin
@cmacmackin
Apr 21 2017 23:17
You are correct. What I do is make all of my large type components dynamic, which happened to be the case for my field types anyway. Static components can not be finalised, hence why I said that this approach doesn't stop all memory leaks. Actually, it gets a little bit more complicated then this, because you can not deallocate components of an intent(in) argument. However, in a non-pure procedure, pointer components of intent(in) objects are a bit odd. It is only required that you don't change the pointer--there is no issue with changing the thing that it points to.
What I did was define an additional, transparent, derived type called called array_1d:
```fortran
  type, public :: array_1d
    real(r8), dimension(:), allocatable, public :: array
  end type array_1d
I have similar types for higher-dimensional arrays. That way I can just deallocate the component array. You can see this in action in the source code.