Mastering Erlang Part 3 – Erlang Concurrent writing multi-module applications

[Giniu 0]

Mastering Erlang part 3 – Erlang Concurrent

Or: Writing multi-module applications

 

Hi again... last time we learned how to create a single module, let's take a look at what we should have/know to continue:

 

Installed Erlang and know how to start and quit from Erlang Virtual Machine (shell).

How to create and what name should have Erlang source files.

How to compile source you just created.

How to create any single text-based module

How to investigate man pages to find structure of build-in functions and standard modules

Have some free time that you want to spend learning even more of Erlang.

 

Let's take you know this all – that part is short, but also very important. First of all I tell you about how to write modules that look good, how to comment module and how you should name your functions and variables... this wasn't important in single module writing, but when it comes to complicated code, it must be readable. So let's start...

 

Programming style in Erlang – or How code should looks like

 

First thing to remember is about attributes – first time I’ve only mentioned them, now I’ll explain them a bit more. Every module must contains attributes and functions, there are many predefined attributes, but you can define your own – those are predefined attributes:

 

-module(Module). - sets name of module, the Module is an atom, same as file name, but without .erl extension, always use lower-case letters to name your modules

 

-export(Functions). - sets what functions would be reachable from outside this module, the Functions is a list, containing functions and numbers of parameters of this function, that are comma-separated, it looks like [Function1/Par1,Function2/Par2,...,FunctionN/ParN].

 

-import(Module, Functions). - allows you to use functions specified in Functions from module Module like they were defined in module where you imported them. Module and Function looks like two attributes above.

 

-compile(Options). - allows you to pass special options to compiler, Options should be list of options, but can be also single option, for complete list of options, check manual of compiler (do erl -man compile).

 

-vsn(Vsn). - vsn stands for version number, it sets version of module. It can be list or number.

 

-behaviour(Behaviour). - sets a description of module behaviour so it can be checked with OTP standards, the standards are: gen_server (generic server), gen_fsm (generic finite state machine), gen_event (generic event manager) and supervisor (main module controlling others). I think I don’t describe them in this tutorial cycle, because they are described in official document called [ OTP Design Principles ] - they for sure make it better than me.

 

Includes, macros and records looks similar, but they will be described in other part of this series, now I’ll tell you only what name they have, so you know what custom attributes names you can use and what you cannot, they are:

 

-include

 

-include_lib

 

-define

 

and

 

-record

 

So – you can define custom attributes that don't have names registered by standard attributes, macros and records. You can define for example attributes like those:

 

-revision('revision code: green').

-created('ad. 2005').

-created_by('Giniu').

 

Every attribute is compiled into beam file, you can reach them from module, using function chunks from module beam_lib, for example, to list them all, do (where Module is atom):

 

beam_lib:chunks(Module, [attributes]).

 

And do find value of specified attribute, use function like that (you must be sure that attribute Chunk exists to use it):

 

find_attribute(Module,Attribute) ->    {ok, {_, [{attributes, Attributes}]}} = beam_lib:chunks(Module, [attributes]),    {value, {Attribute, Value}} = lists:keysearch(Attribute, 1, Attributes),    Value.

 

Last function returns list containing value of attribute, in our example, calling:

 

find_attribute(example,created).

 

returns:

 

[ad. 2005]

 

if you used standard attribute vsn to get it, you can use easier function:

 

beam_lib:version(Module).

 

And get result, like that:

 

{ok, {Module, [Version]}}

 

functions from module beam_lib can be called in module and from outside so they can be used to verify that author of module wasn't changed (while comments aren't included into compiled beam file). So that's why using attributes is important, always place attributes at the beginning of module, module attribute first, then export, next if you must, use import, but try not using input – it makes code harder to read, instead of:

 

-import(Module,[Function/0]). ... Result=Function().

 

use only:

 

Result=Module:Function().

 

Your code would be readable from every part, and when you spot function without module, you see from which module it comes in second and don't have to scroll to top of module. Just after import put vsn and other predefined attributes if you use them. Next all custom attributes. Put empty line between main block (module, export, import), specification block (vsn, compile, etc.) and custom block (all your custom attributes). Keep in mid that if you use extended block (includes/defines/macros) that I will describe in Erlang Everyday (next part) also should be separated from rest of code with empty lines.

 

Now let's go to comments, there are three types of them:

 

module comments – they should be always placed at the beginning of module, start at the beginning of line (without spaces) and with three percent characters (%%%), they should contains general description of module - there is example:

 

%%%--------------------------------------- %%% Key listener manager module for Black Shades game %%%--------------------------------------- %%% This listener waits for signal containing pid of customer process %%% and creates instance of listener that works only for specified module %%% this allows us to create one listener for singleplayer and multiplayer %%% game that works on one machine or through lan/network connection %%%--------------------------------------- %%% Exports %%%--------------------------------------- %%% request_listener(Pid) %%%  creates listener instance that works for process with process id Pid %%%---------------------------------------

 

function comments – (also used for data types and command blocks) they should be always placed before function, start at beginning of line and with two percent characters (%%), they should contains types of values returned and parameters taken, also what is their purpose – here is example:

 

%%---------------------------------------- %% Function: request_listener/1 %% Purpose: create key listener for requesting process %% Arguments: Integer (requesting process id) %% Returns: Tuple ({ok, Pid of listener} or {error, reason}) %%----------------------------------------

 

inline comment – inside comments should be placed at the end of line they refer to, if it isn't possible, they should be placed above that line. They should start with one percent character (%) and be placed in all elements critical for understand of code – here is example:

 

first_function(Argument), % this line refers to function first_function/1 % and this line refers to second_function/2 second_function(Argument1,Argument2).

 

Good comment is very important for good understand of code – you should also provide complete documentation with it – remember to describe all errors with possible reasons, also document containing license, installation, authors, etc., etc., etc. – everything what a user might want to know about it.

 

Now some final rules about designing your code – some hints that may help you in creating large concurrent applications. I’ll try to write them in short points:

Export as few functions as possible

Avoid using import attributes

Comment your code

Don't loop modules (module1 use module2 use module3 use module1) – one crash crashes all

Gather code used in more than one module into library (module containing handy functions)

Don't hide too much messages from user – he might want them to send you bug-report

Start at beginning – put main function at top of module, next just behind it and so on...

Make it run like you want it to and THEN take care to make it run faster (not other way)

Eliminate side effects – make sure you can think about all possible arguments user would type

Make it run always the same – don't allow any random events

Try to handle all possible errors and make module react/restart if needed

Give only one role to process and put only one process into module

Use as much generic functions as possible to make your code portable

Try not to return untagged values (eg. don't use just Value, return {value, Value} instead)

Don't write too much nested code, use recursion

Don't write too large modules, too long functions, too long lines, split them

Choose meaningful variable names, use underscore or large letters to split variables (like: My_variable or MyVariable)

Use function names that is connected with them, use some standard names (like start, stop, init, main_loop), if in different modules there is function that makes the same thing, give it the same name (Module:info()), use underscore to split them (some_function). There are some names that gives hint about return values (is_... -> true|false and check_... -> {ok, ...}|{false, ...})

Use short but meaningful module names, use underscore to split it (like my_math), you might want to simulate hierarchical modules (like: shades_main, shades_listener, shades_listener_key) – I said simulate, because Erlang uses flat module structure (not hierarchical).

Use only one style of writing your modules, for example, when you write tuples in one place like {a,b,c} don't write them somewhere else with spaces, like {a, b, c}

And those are most important things to take care about... extreme care... now you are ready to start our main part of this tutorial:

 

Concurrency in Erlang – or Multi module things...

 

Concurrency is situation, where many processes (not threads – threads shares memory resources, Erlang doesn't share it, so we call them process) running at once. In Erlang concurrency is very easy to obtain, you can create new process using build-in function spawn/3 – it contains module, function and list of function arguments (parameters), like:

 

spawn(Module, Function, List_of_arguments).

 

I’ll explain it on classic example:

 

-module(concurrency). -export([start/0, say_sth/2]).  start() ->    spawn(concurrency, say_sth, ['whats up?', 3]),    spawn(concurrency, say_sth, ['get out!', 3]). say_sth(_, 0) ->    ok; say_sth(What, Times) ->    io:format("~p~n", [What]),    say_sth(What, Times - 1).

 

First of all, take a look how function say_sth works, do:

 

say_sth('whats up?',2).

 

and you get:

 

'whats up?'

'whats up?'

ok

 

but when you type:

 

concurrency:start().

 

You get:

 

'whats up?'

'get out!'

<0.39.0>

'whats up?'

'get out!'

'whats up?'

'get out!'

 

now... what the? So – let me explain... all 'whats up?' came from one process, all 'get out!' from other and <0,39,0> and new line came from function start – exactly it is return value of last function in it – spawn returns PID of created process – which stands for Process ID. Sometimes some function is called too late, then it can looks like:

 

'whats up?'

'get out!'

<0.39.0>'whats up?'

'get out!'

 

'whats up?'

'get out!'

 

Erlang sends messages asynchronous – this means it doesn't wait for receive and doesn't care if process is running. Module sends request and don't thinks about it's targets (modules that depends on it) – programmer must himself implement whole system that would check if message was delivered. This is very important aspect of concurrent programming in Erlang. When I would be giving you example results I would consider processes made it in time (first example, without empty line). So get back to PID's. Every process has a PID, you can get it using function:

 

self().

 

also you can automatically receive PID of created process by spawn return value. So expand our example a little bit...

 

-module(concurrency). -export([start/0, say_sth/2]).  start() ->    Pid=self(),    Pid1=spawn(concurrency, say_sth, ['whats up?', 3]),    Pid2=spawn(concurrency, say_sth, ['get out!', 3]),    io:format("~w says: ~w (whats up) and ~w (get out) created.~n", [Pid, Pid1, Pid2]). say_sth(_, 0) ->    io:format("~w says: finished.~n", [self()]); say_sth(What, Times) ->    io:format("~w~n", [What]),    say_sth(What, Times - 1).

 

Here is how it looks like after execution of function start():

 

<0.30.0> says: <0.131.0> (whats up) and <0.132.0> (get out) created.

'whats up?'

'get out!'

'whats up?'

'get out!'

ok

'whats up?'

'get out!'

<0.131.0> says: finished.

<0.132.0> says: finished.

 

As you see process can you tell it's PID. You can also communicate them using their PID's. Back to example – the “ok” line comes from io:format not followed by any command. Now I’ll tell you something about sending messages between processes, the syntax of it is very easy:

 

Pid ! Message

 

sends Message to process Pid. This send structure also returns Message as it's return value. The process that created a new one is called a parent of this process and the created process is it's child.

 

When the process have to wait for messages, the receive structure is used:

 

 

receive

Pattern1 ->

Action1,

Action1,

Action1;

Pattern2 ->

Action2;

Pattern3 ->

...

PatternN ->

ActionN

end.

 

The receive structure returns same value as Actions that are executed in it. To call some process you must remember it's Pid – but sometimes it is hard to remember all of them – we can (should) register all used Pid's – this also is very easy:

 

register(Alias, Pid).

 

where Alias is an atom that is be used to recognize process number Pid. And then we can send it for example, like:

 

name_of_process ! Message

 

So – now some small theory of messages. All sent messages are sent and are stored in process mailbox until they can be read. They are read in order of income, but they must fit given pattern, if they won't, they’ll wait till other receive. Sometimes it is important to get response – then you must send address to which process should respond, there is standard message structure that is very easy – it is tuple {Pid, Message} so process knows from where message come.

 

There is example how to create a classic – simple echo process... one process waits for message for other and that one would wait for response, then would confirm finish of listening of second process:

 

-module(echo). -export([say/1, listener/0]).  say(Word) ->    Pid = spawn(echo, listener, []),    Pid ! {self(), Word},    io:format("~w: \'~w\'~n", [self(), Word]),    receive       {Pid, Message} ->          io:format("~w: ~w~n", [Pid, list_to_atom(Message)])       end,       Pid ! stop. listener() ->    receive       {From, hello} ->          From ! {self(), "Nice to see you..."},          listener();       {From, _} ->          From ! {self(), "Sorry, I do not know what that means."},          listener();       stop ->          true    end.

 

You can now run it passing a hello atom or anything else, so you gets:

 

echo:say(hello).

<0.30.0>: 'hello'

<0.47.0>: 'Nice to see you...'

stop

 

or:

 

echo:init(helloa).

<0.30.0>: 'helloa'

<0.47.0>: 'Sorry, I do not know what that means.'

stop

 

the receive command waits for message that match any pattern – it executes commands which are connected with this pattern, and continue execution behind receive block. In our example, function listener after we get some message is restarting itself to wait for other messages (if we aren't sending message stop, so it is returning only value true to tell that execution was successful) so we can turn it off when we want and send as many messages as we want.

 

Good design in message sending is a key to success with concurrency and distribution in Erlang or any other languages.

 

Now some exercise – if it is too hard or I don't described those methods enough just let me know and I’ll add needed things to this part (maybe more examples or something) but I think that if you was making all exercises before – you shouldn't have any troubles. Let's go to training... This time we will work a little different – first of all – concurrent programs often are first drawn on piece of paper – the circle means a process and the arrows shows communications way. From this the name of some schemes taken their names. You’ll make a “yo-yo”. This name isn't very popular, sometimes this scheme is called a line, but I think that “yo-yo” is more descriptive. Let me explain how it works – your program creates a process that creates a process and so on, then when it reaches the length of yo-yo that is specified, the last process sends a close signal to it's parent and quits. When any process gets close signal it must send same signal to it's parent and quit. So – It looks like yo-yo when you would draw this vertical.

 

Create a module that does that, it must have a function init/1 that takes a length of yo-yo (number of process to create excluding the one that you started). It must inform about creation of every process with: “Process <PID> created process <PID>” and every finished process “<PID> will be closed now...” also when process receives a close signal “<PID> knows that <PID> finished”. Also when you start the execution inform user: “Yo-yo started” and the last line should be “Yo-yo returned”. After execution you shouldn't have any process running.

 

Hint: in every process store Pid of it's parent and number of process created so it knows if it should create other process or start turning off, pass that number from process to process.

 

If you don't know how you can do it, PM me and I’ll send you hints – but write with what you had troubles so I’ll only show you the way to go... this is an exercise – not an example

 

--------------------------

Thanks for correction of this tutorial goes to Nelle... Thanks again!

 

------------------------------------------

changes since first version? - No...