1 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
3 % NEPI, a framework to manage network experiments
4 % Copyright (C) 2013 INRIA
6 % This program is free software: you can redistribute it and/or modify
7 % it under the terms of the GNU General Public License as published by
8 % the Free Software Foundation, either version 3 of the License, or
9 % (at your option) any later version.
11 % This program is distributed in the hope that it will be useful,
12 % but WITHOUT ANY WARRANTY; without even the implied warranty of
13 % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
14 % GNU General Public License for more details.
16 % You should have received a copy of the GNU General Public License
17 % along with this program. If not, see <http://www.gnu.org/licenses/>.
19 % Author: Alina Quereilhac <alina.quereilhac@inria.fr>
21 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
23 NEPI is written in Python, so you will need to install Python before
24 before being able to run experiments with NEPI.
25 NEPI is known to work on Linux (Fedore, Debian, Ubuntu) and Mac (OS X).
27 \section{Dependencies}
29 Dependencies for NEPI vary according to the features you want to enable.
30 Make sure the following dependencies are correctly installed in your system
33 Mandatory dependencies:
39 Optional dependencies:
41 \item SleekXMPP - Required to run experiments on OMF testbeds
44 \subsection{Install dependencies on Debian/Ubuntu}
47 \fontsize{10pt}{12pt}\selectfont
50 $ sudo aptitude install -y python mercurial
55 \subsection{Install dependencies on Fedora}
58 \fontsize{10pt}{12pt}\selectfont
61 $ sudo yum install -y python mercurial
66 \subsection{Install dependencies on Mac}
68 First install homebrew (\url{http://mxcl.github.io/homebrew/}),
72 \fontsize{10pt}{12pt}\selectfont
80 \subsection{Install SleekXMPP}
82 You will need \textit{git} to get the SleekXMPP sources.
85 \fontsize{10pt}{12pt}\selectfont
88 $ git clone -b develop git://github.com/fritzy/SleekXMPP.git
90 $ sudo python setup.py install
95 \section{The source code}
97 To get NEPI's source code you will need Mercurial version
98 control system. The Mercurial NEPI repo can also be browsed online at: \\
100 \url{http://nepi.inria.fr/code/nepi/}
102 \subsection{Clone the repo}
105 \fontsize{10pt}{12pt}\selectfont
108 $ hg clone http://nepi.inria.fr/code/nepi -r nepi-3.0-release
113 \section{Install NEPI in your system}
115 You don't need to install NEPI in your system to be able to run
116 experiments. However this might be convenient if you don't
117 plan to modify or extend the sources.
119 To install NEPI, just run \emph{make install} in the NEPI source
123 \fontsize{10pt}{12pt}\selectfont
132 If you are developing your own NEPI extensions, the installed
133 NEPI version might interfere with your work.
134 In this case it is probably more convenient to tell
135 Python where to find the NEPI sources, using the PYTHONPATH
136 environmental variable, when you run a NEPI script.
139 \fontsize{10pt}{12pt}\selectfont
142 $ PYTHONATH=$PYTHONPATH:<path-to-nepi>/src python experiment.py
147 \section{Run experiments}
149 There are two ways you can use NEPI to run your experiments.
150 The first one is writing a Python script, which will import
151 NEPI libraries, and run it.
152 The second one is in interactive mode by using Python console.
154 \subsection{Run from script}
156 Writing a simple NEPI expeiment script is easy.
157 Take a look at the example in the FAQ section \ref{faq:ping_example}.
158 Once you have written down the script, you can run it using
159 Python. Note that since NEPI is not yet installed in your system,
160 you will need to export the path to NEPI's source code to
161 the PYTHONPATH environment variable, so that Python can find
165 \fontsize{10pt}{12pt}\selectfont
168 $ export PYTHONPATH=<path-to-nepi>/src:$PYTHONPATH
169 $ python first-experiment.py
174 \subsection{Run interactively}
176 The Python interpreter can be used as an interactive console to execute
178 We can take advantage of this feature, to interactively run experiments
179 with NEPI. We use the \textit{ipython} console for the example below,
180 you can install it with \textit{sudo yum/aptitude install ipython} on
184 \fontsize{10pt}{12pt}\selectfont
187 $ export PYTHONPATH=<path-to-nepi>/src:$PYTHONPATH
189 Python 2.7.3 (default, Jan 2 2013, 13:56:14)
190 Type "copyright", "credits" or "license" for more information.
192 IPython 0.13.1 -- An enhanced Interactive Python.
193 ? -> Introduction and overview of IPython's features.
194 %quickref -> Quick reference.
195 help -> Python's own help system.
196 object? -> Details about 'object', use 'object??' for extra details.
202 With ipython, if you want to paste many lines at once you will need to type
203 \emph{\%cpaste} and finish the paste block with \emph{\-\-}.
205 The first thing we have to do is to import the NEPI classes
207 In particular we need to import the ExperimentController class.
209 \begin{lstlisting}[language=Python]
211 from nepi.execution.ec import ExperimentController
215 Then we need to create an ExperimentController instance.
216 The \textit{exp-id} argument serves as a human readable identifier
217 for the experiment to be ran.
219 \begin{lstlisting}[language=Python]
221 ec = ExperimentController(exp_id = "<your-exp-id>")
225 Next we will define two Python functions, one to register \emph{LinuxNode}
226 resources and the other to register \emph{LinuxApplication} resources.
227 A \emph{LinuxNode} resource (or ResourceManager) will serve as an abstraction
228 to a Linux host resource, that can be accessed using SSH key authentication.
229 A \emph{LinuxApplication} resource represents anything that can be executed
230 on a Linux host as a BASH command.
232 \begin{lstlisting}[language=Python]
235 def add_node(ec, host, user, ssh_key):
236 node = ec.register_resource("LinuxNode")
237 ec.set(node, "hostname", host)
238 ec.set(node, "username", user)
239 ec.set(node, "identity", ssh_key)
240 ec.set(node, "cleanHome", True)
241 ec.set(node, "cleanProcesses", True)
244 def add_app(ec, command, node):
245 app = ec.register_resource("LinuxApplication")
246 ec.set(app, "command", command)
247 ec.register_connection(app, node)
253 The method \textit{register\_resource} declares a resource instance to the
254 Experiment Controller. The method \textit{register\_connection} indicates
255 that two resource will interact during the experiment.
256 Note that invoking \textit{add\_node} or \textit{add\_app} has no effect other
257 than informing the EC about the resources that will be used during the experiment.
258 The actual deployment of the experiment requires the method \textit{deploy} to
261 The Resource Managers (RM) that abstract the concrete resources expose
262 configuration attributes. In the LinuxNode RM we set the \emph{hostname}
263 and \emph{username} as attributes, while in the LinuxApplication RM
264 we set the \emph{command} attribute.
266 Apart from teh \emph{command} attribute, the \emph{LinuxApplication}
267 ResourceManager exposed several other attributes
268 that permit to upload, compile and install arbitrary sources,
269 to run any application might be needed to run an experiment.
270 More details will be given in the following sections of this document.
272 Lets now use these functions to describe the experiment we will run.
273 Choose a host where you have an account, and can access using SSH
274 key authentication. Define string variables with the right
275 values for the \emph{hostname}, \emph{username} and path to the
276 SSH public key in as \emph{ssh\_key}, and then type the following lines.
278 \begin{lstlisting}[language=Python]
280 node = add_node(ec, hostname, username, ssh_key)
281 app = add_app(ec, "ping -c3 nepi.inria.fr", node)
285 Now that we have described our simple PING experiment, we can deploy
286 it. Invoking the \emph{deploy} command will not only configure the
287 resource but also automatically launch the applications.
289 \begin{lstlisting}[language=Python]
295 After some seconds, we should see some log messages informing about
296 the progress of the deployment.
297 If you now open another terminal and connect to the host through SSH,
298 you should find some directories created by NEPI.
299 You should see a directory named \emph{nepi-exp}, and under that directory
300 you will find another with the identifier you gave when you created the
301 experiment controller (the <exp-id>).
302 Under the experiment directory, you will find directories for each of the
303 resources deployed (e.g. \emph{node-1}, \emph{app-2}).
304 The resource directories are named with a short string that identifies the
305 type of resource (e.g. 'node', 'app', etc), followed by a unique number that
306 uniquely identifies a given resource within the experiment,
307 the global unique identifier (guid).
309 From the ipython console, we can check the deployment status of each resource
310 by querying the EC with the method \emph{state}.
311 The argument \emph{hr} stand for `human readable`, and will return a string
312 state instead of a state number.
314 \begin{lstlisting}[language=Python]
316 ec.state(app, hr=True)
320 Once the LinuxApplication is STARTED, we can retrieve the PING output using
321 stored as a \emph{trace} file on the host. For this we use use \emph{trace}
322 method, and specify the resource and the trace name (i.e. stdout).
324 \begin{lstlisting}[language=Python]
326 ec.trace(app, "stdout")
330 That is it. We can terminate the experiment by invoking the method \emph{shutdown}.
332 \begin{lstlisting}[language=Python]