FIFE forums

Please login or register.

Login with username, password and session length
Advanced search  

News:

FIFE 0.4.0 has been released on 15th of January, 2017!

Author Topic: Scripting APIs, documentation and memory management  (Read 10651 times)

jasoka

  • Developer
  • Jr. Member
  • *
  • Posts: 51
    • View Profile
Scripting APIs, documentation and memory management
« on: October 19, 2007, 06:40:05 pm »

Hi all,

Techdemo starts to be in such shape, that it gives a first glimpse of possible engine scripting APIs: https://mirror1.cvsdude.com/trac/fife/engine/browser/trunk/techdemo.py?rev=1733. At this point of time it could be a good opportunity to stop and think for a while where we want to go with this. Even though the current python code is quite readable (personal opinion), we clearly could do better. To prove this, I tried to write  the logic of techdemo in more fluent way using more pythonic idioms. The result can be seen here: http://pastebin.com/fb9e00c0.

As you probably guessed, the code cannot be executed at the moment due to heavy api changes. Therefore there's naturally also some bugs & typos there. Anyway I hope it makes my thoughts clear, and you agree that a) code amount is much smaller. b) it is easier to understand and c) it contains the same functionality.

The problem with the example is that such functionality is hard or even impossible to reach with plain swig interfaces. Therefore I propose that we should consider the possibility of writing still another wrapper layer on top of most of the swig apis with scripting languages (meaning the extension layer, see http://wiki.fifengine.net/index.php?title=Architecture_Documentation#High_Level_Layering)

Another point with scripting apis is documentation. Despite of my search, I haven't (at least yet) found any good way to auto-generate documentation from swig interface files. Furthermore if we write extensions on top of that, we should be able to mix their documentation with the documentation from swig generated interfaces. Properly documented apis are probably one of the most important factors when thinking about possible adoption of fife for game development purposes.

On top of that we still have engine memory management issues. I have written a wiki article pointing some of the problem aspects: http://wiki.fifengine.net/index.php?title=Engine_Memory_Management. Any comments on that area are more than welcome.

With previous points + techdemo example in mind, I would like to point out the following:
  • The need for extension wrapping increases along with the need for more fluent interfaces
  • Generating documentation for plain python interfaces is easy, see e.g. epydoc http://epydoc.sourceforge.net/
  • Combination of some consistent memory management strategy AND extension layer on top of SWIG interfaces would probably give us the best alternative to provide clean memory management design for games using FIFE
  • Extension wrapping would give us additional buffer to keep our APIs, after we have stabilized them a bit
  • Wrapping would also add our code amount and make it harder to port fife to other scripting languages  :-[

Despite the last bullet, my current feeling is that we should really look into our scripting apis at this point of development and increase the size of extensions. I would also like to hide most of the plain swig interfaces from games, and provide properly documented and easy to use alternatives to them from extension layer.

Any thoughts?


Logged

chewie

  • Developer
  • Full Member
  • *
  • Posts: 123
    • View Profile
    • zero-projekt.net
Re: Scripting APIs, documentation and memory management
« Reply #1 on: October 20, 2007, 07:57:21 am »

@gui api

I like the idea of the API for the gui, should save many lines of code, indeed.  :)

BTW - run_world() & main() should be more generalized - ATM it is designed for loading one map - but what about different maps / mapchanges?

Or do we use only one equal name for all the tile layers? Then we could leave most parts of run_world() as it is...

Code: [Select]
camloc = self.map.get_location(layer_id='TechdemoMapTileLayer', (5, -1))


Start location (5, -1) should come from the map itself, right?

Code: [Select]
<metadata>
<param name="_START_POSITION" type="Point"><x>5</x><y>-1</y></param>
</metadata>

I'd also say we should move tilt and rot to the map, too. (e. g. When using a topdown map as worldmap / travel map, tilt and rot is different)

Code: [Select]
<metadata>
<param name="_rot" type="int"><item>35</item></param>
        <param name="_tilt" type="int"><item>60</item></param>
</metadata>

jasoka

  • Developer
  • Jr. Member
  • *
  • Posts: 51
    • View Profile
Re: Scripting APIs, documentation and memory management
« Reply #2 on: October 20, 2007, 09:00:08 am »

BTW - run_world() & main() should be more generalized - ATM it is designed for loading one map - but what about different maps / mapchanges?
Yes, the script structure will probably change along the new functionality. Refined script just converted the existing functionality to new syntax to demonstrate the need for refinement. Refining periodically is the process that I would like to use with the techdemo in general; create a little, refine a little, create a little more and then refine again.

Start location (5, -1) should come from the map itself, right?
...
I'd also say we should move tilt and rot to the map, too. (e. g. When using a topdown map as worldmap / travel map, tilt and rot is different)
Those attributes should indeed prolly be in xml. Same could possibly be used for gui in the future (like phoku has demonstrated before).
Logged

jwt

  • Developer
  • Newbie
  • *
  • Posts: 26
    • View Profile
Re: Scripting APIs, documentation and memory management
« Reply #3 on: October 20, 2007, 01:50:22 pm »

I mostly agree with all of this. My only concern is that this would tie us rather strongly to python scripts (as jasoka has pointed out). I don't think this is necessarily a bad thing. Python is an easy-to-use, mainstream language with lots of support and libraries. I also don't think multiple scripting language support is necessarily good, even if it was possible. If different teams were all writing games in different languages, I think it would be difficult to develop a strong FIFE community.

I think if we go this route, we should be honest about scripting opportunities. Already enough of FIFE's features (loaders/savers) are python-side that it would be difficult to use a different scripting language (or pure c++, as a person in irc was recently attempting). If we agree to develop much more code in python, then we should not even mention the possibility (or more likely impossibility) of using a different language.
Logged

anxs

  • Newbie
  • Posts: 14
    • View Profile
Re: Scripting APIs, documentation and memory management
« Reply #4 on: October 21, 2007, 11:31:14 am »

First let me say that I think the additional abstraction layer is basically a very good idea, but I also have to agree with jwt. It might be difficult to provide a wide range of languages.

I think it's very important to fully support pure C++ usage of FIFE. Although python is a very powerful and easy language, many game devs might want to create their game on a pure c++ basis. (because they don't know anything about python or don't trust in it's performance)

To support multiple scripting languages on the one hand might not make sense, because one powerful scripting-language is enough. But on the other hand we develop FIFE - A Flexible Fallout-like Engine and therefore should support as many languages as possible. ;)

Nevertheless we'll need a consistent memory management solution and my personal choice would be the memory manager.
Logged

jasoka

  • Developer
  • Jr. Member
  • *
  • Posts: 51
    • View Profile
Re: Scripting APIs, documentation and memory management
« Reply #5 on: November 10, 2007, 06:34:01 am »

small update on the issue. I tried to check how wxWidgets is implementing memory management, and encountered this:

...
// This is called when an OOR controled object is being destroyed.  Although
// the C++ object is going away there is no way to force the Python object
// (and all references to it) to die too.  This causes problems (crashes) in
// wxPython when a python shadow object attempts to call a C++ method using
// the now bogus pointer... So to try and prevent this we'll do a little black
// magic and change the class of the python instance to a class that will
// raise an exception for any attempt to call methods with it.  See
// _wxPyDeadObject in _extras.py for the implementation of this class.

-> search for _wxPyDeadObject from google code

Looks like we might be able to protect scripts from invalid memory addresses without smart pointer approach after all.
Logged

neurogeek

  • Newbie
  • Posts: 3
    • View Profile
Re: Scripting APIs, documentation and memory management
« Reply #6 on: December 03, 2007, 08:35:12 am »

Hello,

   Im new to FIFE, and im trying to get involved to help a bit.

   Regarding the issues addressed here I have some points that might help.

   First, about the high layering of scripting APIs, i don't necessarily agree with the fact that this approach should tie us to Python. Although im a Python fan, this means that FIFE should concentrate on a scripting language to advance into other stuff, but if i get this right, that leaves other people to write their own high layer APIs for other scripting languages. At the end, thats what OpenSource really means. If we adopt Python as a language for developing the needed APIs but maybe re-write or clean SWIG interfaces so they could be easily ported to another scripting languages, you guys are not killing the possibility of adding support for other languages. Although im not a great fan of having that many ingredients on a recipe.

   Regarding the memory management issue, it is well known that Python's "destructors" are not that reliable, less when you add Swig to the equation. So, taking advantage of Python's reflection and flexibility, thats not a bad idea of changing Python classes in runtime so they raise exceptions when swig pointers stop being valid.

Logged

neurogeek

  • Newbie
  • Posts: 3
    • View Profile
Re: Scripting APIs, documentation and memory management
« Reply #7 on: December 18, 2007, 09:31:57 am »

Hello..

   Regarding this issue.. please refer to http://mirror1.cvsdude.com/trac/fife/engine/ticket/197

Best regards
Logged

jasoka

  • Developer
  • Jr. Member
  • *
  • Posts: 51
    • View Profile
Re: Scripting APIs, documentation and memory management
« Reply #8 on: January 12, 2008, 09:55:52 am »

Thanks, neurogeek. Any plans to develop this solution even further?
Logged

Wuntvor

  • Developer
  • Newbie
  • *
  • Posts: 9
    • View Profile
Re: Scripting APIs, documentation and memory management
« Reply #9 on: January 17, 2008, 12:14:25 pm »

Has this ever really been decided on? Just curious, from what I've seen FIFE seems to be deeply integrated with python (judging by the amount of python code for stuff like the GUI), but I'd really like to know.
Logged

jasoka

  • Developer
  • Jr. Member
  • *
  • Posts: 51
    • View Profile
Re: Scripting APIs, documentation and memory management
« Reply #10 on: January 17, 2008, 12:26:10 pm »

Has this ever really been decided on?
hmm... depends on what you are asking? :)

"Deeply integrated" statement can be questioned, since it is in fact so that fife core is extended with python scripts. Check out the current specs for the engine (draft still): http://wiki.fifengine.net/index.php?title=Architecture_Documentation
Logged