나는겁쟁이

I'm spikeekips, just develop something.

Embedding Python In Golang

In these days, I am stick to the new private project, to find the way to run python code inside Golang runtime. This is the story I experienced to struggle with Python and Golang.

As you may know, I have some experience in python programming, exactly to say, I love Python, easy to learn, especially in python, code explains itself without any special comment, for many years until now, python would be the first choice for my new projects.

Python will be the great language, but it’s not at this time, because it’s performance is not good enough for manysome area, e.g.

  • heavy-loaded network server like web server.
  • long running tasks
  • handling massive data
  • critical tasks of memory
  • etc.

I thought, if possible to combine Golang and python, it can bring the extreme way to me, getting the productivity from python and performance from Golang simultaneously.

Why golang, Instead Of C

Using C extension is well known and most reasonable consideration to extend python. If we can extend python with other language, it will be benefit for both side.

The reason to select golang within many other languages, I love Golang and it’s goroutine. Golang will be the best /* alternative */ language for python programmers, who miss the static typing and performance.

Golang is the fast growing in the language world, the brilliant features like goroutine and channel, etc will stimulate thenother languages to focus in their perfornance improvement.

The Bottom Line First.

After all, I dumped the latest proof-of-concepts in embedding-python-in-golang in github, I introduced the several methods to embed python in Golang.

The final goal will be to run the django and flask application with WSGI interface inside the net/http of Golang.

Story to find the way.

At first I tried to find the similar attempts and there are already similar project,

gopy seems to be prominent, it can make python extension in golang and go-python implements the C-API of python in golang, but it does not provide some key API like Py_InitModule.

I wanted to get result as soon as possible, so I started with gopy.

2 days with gooy

At first, I tested gopy. gopy can make Golang code to python extension, so I thought,

  1. Write golang code for python extension
  2. Build python extension with gopy
  3. Import and call the golang function

At this time, I wanted to run golang code in python, because I just want to combine Golang and python adhesively. Whether which language will be main runtime does not matter.

But one thing I missed, the gopy project was not maintained in these days, and it does have some bugs, e.g. NewString does not return the string of Golang and several other bugs made me so to be hesitate to use this library.

I needed another way, that was go-python.

And go-python

go-python follows the same strategy with the gopy, uses python C-API, but the differences comes, it was still maintained and is the good example of cgo and python C-API.

Unlike gopy, go-python does not support to create python extension sadly, so I need to change the way, run python code inside golang.

After all, go-python works well except GIL problem in python, but this does not come from golang as we know.

In the next post, I will explain how to solve the the GIL problem.