X-Git-Url: http://git.onelab.eu/?a=blobdiff_plain;f=README;h=9dc93feb84ff06d638e1807edcaba24892bf8a24;hb=26b2d32f6132a3e619f47411adeccfa693b71162;hp=52c5e4199b801c4b3ecca76f4d68f04ae906a159;hpb=8efc599bc1c20bfd8172e96a7e27f196d08adf47;p=myslice.git diff --git a/README b/README index 52c5e419..9dc93feb 100644 --- a/README +++ b/README @@ -1,96 +1,178 @@ -This file documents the contents of this module +Last update 18 MAR. 2015 + +Installation +================================================================= +Complete Guide: http://trac.myslice.info/wiki/Manifold/Install + +Recommended OS +=============== +Debian GNU/Linux 7.5 (wheezy) x64 + +PYTHON DEPENDENCIES +===================== +sudo apt-get install python-pip or sudo easy_install pip==1.4.1 +sudo apt-get install python-dev (for paramiko and pyOpenSSL) +sudo apt-get install libffi-dev (for pyOpenSSL) + +$ pip install django=="1.5.2" +$ apt-get install python-django-south +$ pip install requests +$ pip install djangorestframework +$ pip install django-celery +$ pip install geopy +$ pip install paramiko +$ pip install pyparsing +$ pip install python-dateutil +$ pip instal pyOpenSSL + +MYSLICE +======= +git clone ssh://yourlogin@git.onelab.eu/git/myslice.git +cd myslice +git checkout onelab + +edit/create myslice/myslice.ini and enter the details of your manifold backend + +$ ./manage.py syncdb +$ ./manage.py migrate +$ ./manage.py collectstatic + +Run the development server +$ ./devel/server-loop.sh + +MANIFOLD +========== +git clone git://git.onelab.eu/manifold.git +cd manifold +git checkout devel +make && make install + +SFA +=== +$ git clone -b geni-v3 git://git.onelab.eu/sfa.git +$ cd sfa +$ git checkout geni-v3 + +$ make version +$ python ./setup.py install + +===================================================================== + +## Whenever doing a git pull the following operations are recommended: + +$ make static # will refresh static/ from all the pieces in the project +$ make templates # same, for templates +$ make redo-static # clears up (rm -rf) static/ first, then make static +$ make redo-templates # ditto for templates +$ make redo == make redo-static redo-templates -See the devel/ subdir for more devel-oriented doc. -==================== 1 minute howto - -* REQUIREMENTS is to have python + django installed django -** should be straightforward -** see devel/django-install.txt in case of trouble +* run a local server: +$ ./manage.py runserver 0.0.0.0:8000 +-- or -- my advice: +$ devel/server-loop.sh +when you just need to hit ^C yourself when your static files need to be refreshed - see below -* git clone git://git.onelab.eu/myslice-django.git --- or -- -* git clone ssh://yourlogin@git.onelab.eu/git/myslice-django.git +* use it from your browser +(See more notes on using the development server below) -* edit myslice/settings.py and -** change DEVELOPER_ROOT if you didnt clone into ~/git/myslice-django -** change the location of your backend API (not yet supported) +* install dependencies +$ pip install -r path/to/requirements/file.txt +Note. not quite sure what this is about, I could not spot this file.. -* init django -$ manage.py syncdb +=====APACHE Config===== +if are running apache please do: -[ at this point point it *might* be needed to run -$ make allst - but as far as running a local server is concerned you should be good without that - feedback on this is appreciated] +$ chown www-data:www-data /var/myslice-f4f -* run a local server: -$ manage.py runserver 0.0.0.0:8000 +otherwise you may get the following error: +"DatabaseError: unable to open database file" -* use it from your browser +==================== Status +*** Authentication *** -==================== Status +Should be mostly fine +Not quite sure if/how the user gets proper notifications when +. his session has expired (i.e. his frontend is not logged into the backend any longer) +. his credentials have expired (i.e. the uploaded credentials, e.g. SFA delegated cred) + expired and she needs to run e.g. sfi myslice again -For now there's not much in here; +Hard-coded accounts (from a very early stage) are gone -* Logging in should be easy using one of the few hard coded accounts - this was the simplest possible way to demonstrate deferring authorization elsewhere -* To logout, click on 'logged as *jean*', this shows a confirmation page.. +*** Packaging *** -Right now the UI has 4 views: +I've done a very rough attempt at packaging for rpm. +The logic seems about right but needs more work, in particular in terms of installing myslice.conf +in the httpd conf.d directory. +It seems like our app won't work on f14 as is because Django is only 1.3.1 on f14 +Plan is to target f18 but I lack a test machine. +Also of course I'll try to tackle debian/ubunti at some point. -* the most useful one being the 'Plugin' view which demonstrates all the available plugins - see test_plugin_view in engine/views.py +There also is a working packaging for debian(s) and ubuntu(s) that we use +on an almost daily basis to upgrade manifold.pl.sophia.inria.fr -* slice view : only demonstrates how to use URLs to pass arguments along -* scroll view : forget about that one, but it does illustrate that some pages can be made public (no need to login) -* tab view : a hand-made tab widget -(last three from slice/views.py) +==================== Third party tools shipped: * jquery * datatables +* spin * bootstrap -not much effort has been put into coming up with a nice layout and all, feel free to tweak that but it's probably way too early for that +Others are added as we build the system when they become needed +Look in third-party/ for a more detailed list + +As a rule of thumb, please try to keep in mind that these will need to +be upgraded over time I've tried to keep track of the version I picked +and to have an easy upgrade path (depending on the way the original +package is published) ==================== Contents: 1st level subdirs ========== code from git -* myslice: + +* myslice/ this is the django 'project', where to look for . settings.py . urls.py -* engine: - the code for building / rendering plugins +* manifold/ + the code for dealing with queries, sending them to the backend, and offering the /manifold/proxy/ URL -* plugins: +* unfold/ + the code for building / rendering plugins + +* plugins/ the actual code for plugins -* auth: +* auth/ a django 'app' that deals with authentication; see especially auth.backend.MyCustomBackend for how to use a separate authentication system, as well as settings.py for how to enable it -* slice: - a django app for dealing with slice views +* ui/ + provides building blocks for the UI, especially layouts (1 or 2 columns) as + well as the topmenu widget + + some global static files (css, js, images..) + +* portal/ + this is where the first implementation of myslice, with complete + user-management including registration, is taking place + +* trash/ + rough/preliminary scaffolding views are in here + as the name suggests this is temporary * insert_above: a third-party django app for adding on-the-fly mentions to css or js files that need to go in the header -* templates/ - * some global templates (django templates, i.e. (essentially html) fragments that can be specializied) - more on this below - -* static/ - * third party stuff (bootstrapfs, jquery, datatables) - * + some global static files (css, js, images..) +* third-party/ + * third party javascript and css stuff (bootstrapfs, jquery, this kind of things) see more about that below too * devel: @@ -98,64 +180,97 @@ not much effort has been put into coming up with a nice layout and all, feel fre ========== automatically generated -* all-static: (generated, no need to source-control) - this is where 'manage.py collectstatic' will gather all your static contents if you run a local server - make has convenience targets to refresh this area - $ make clean-static - $ make static +* static/: (generated by collectstatic, see above, do not source-control) + $ manage.py [ --noinput ] collectstatic + +* templates/ * myslice.sqlite3 this is where django stores its own stuff, as per settings.py +==================== conventions for templates & static files +==================== and NOTES on using the development server + +. first off, running manage.py runserver is provided by django as a development convenience but + SHOULD NOT be used in production + +. second, when you do use it for developement purposes, please be aware that: + +.. the recommended layout for the various files and pieces (py, html, js and css) with django is e.g. + plugins/quickfilter/___init__.py, + plugins/quickfilter/templates/quickfilter.html, + plugins/quickfilter/static/js/quickfilter.js + plugins/quickfilter/static/css/quickfilter.css + plugins/quickfilter/static/img/some-image.png + +.. the files actually used by the development server are the ones located in + static/ + templates/ + +you can and should use the following make targets to refresh the +contents of these directories when running a developement server +$ make static to refresh static/ +$ make redo-static to clean up static/ and then refresh its contents +$ make templates to refresh templates/ +$ make redo-templates to clean up templates/ and then refresh its contents +$ make redo equivalent to make redo-static redo-templates + +.. as far as possible, please make sure to use third-party to store +any javascript tool or utility that your plugin depends upon + +also we have the convention that all material in third-party should be +tagged with a version number, with a symlink pointing to the version +being used, like this + +~/git/myslice/third-party $ ls -ld spin* +lrwxr-xr-x 1 parmentelat staff 10 Sep 6 17:55 spin -> spin-1.3.0 +drwxr-xr-x 7 parmentelat staff 238 Sep 6 17:55 spin-1.2.8 +drwxr-xr-x 7 parmentelat staff 238 Sep 6 17:55 spin-1.3.0 + +finally, as far as possible we keep track of the urls used to pull +stuff in the first place so that upgrades are easier + +. third, be careful when importing third party material, to stay away from demo-oriented material + +e.g. at some point we were using demo_page.css and demo_table.css from the datatables demo and sample pages +unfortunately these are not tailored for production use as they are setting styles on a very wide scope +that breaks a lot of stuff, so please refrain from using these altogether -==================== conventions for templates and static -* templates: - we store this under templates/ within the corresponding app, e.g. - auth/templates/login.html - for now this is mostly about html, but the engine can be used for rendering anything including js(on) or whatever.. +======== update django database to reflect changes in existing models without any migration system (e.g., south) ========= -* static files: - we chose to have all static files (images, but also javascript and stylesheets) in the various - proj or app where they belong, under a static/ subdir that has this structure: - where-it-belongs/ - static/ - img/ - css/ - js/ +# older version +$python manage.py reset -NOTE. in essence any of these files can be moved around (from e.g. one templates/ dir to another) -at any time without any change needed in the code +#Django 1.5.1 or later +$python manage.py flush -* filenames / locations +This will update the database tables for your app, but will completely destroy any data that existed in those tables. +If the changes you made to your app model do not break your old schema (for instance, you added a new, optional field) +you can simply dump the data before and reload it afterwards, like so: - you can run the following make targets to have a summary of where things are +$python manage.py syncdb +$python manage.py dumpdata > temp_data.json +$python manage.py flush +$python manage.py loaddata temp_data.json - $ make list-html list-js list-css list-img - -- or -- - $ make list-all +If your changes break your old schema this won't work - in which case tools like south or django evolution are great. - as far as possible it's great to have things named the same after e.g. say a plugin name - like e.g. - plugins/quickfilter.py - plugins/templates/quickfilter.html - plugins/static/js/quickfilter.js - plugins/static/css/quickfilter.css - it's sometimes not working though, so here are a few additional conventions +Add a new model to the DB - the global layout (for now I needed only one) is named layout-myslice.html - it provides the global layout with one menu stuck on top and 2 divs 3/4 1/4 +$python manage.py schemamigration --auto +$python manage.py migrate - the views that I currently have are all named in templates/view-<>.html - they all extend a layout - they should/could admittedly move where they belong (in auth/slice/engine) - so that they would go with their views.py code, but for now it's impler this way +======== update django database to reflect changes in existing models with migration system (e.g., south) ========= - and there are 3 'widgets' (login, logout, and topmenu) - these could maybe be best implemented as a plugin now that I have the right model for that - however esp. for the login/logout I'm not quite sure this will remain over time - so for now it's good enough like this +As south is already installed , you just have to do: -So in summary there's still room for a lot of improvement on this particular topic.. -At the very least topmenu should be made a plugin, and maybe also login/logout +# ./manage.py schemamigration portal --initial +# sqlite3 /var/unfold/unfold.sqlite3 + sqlite> DROP TABLE "portal_institution"; + sqlite> DROP TABLE "portal_pendinguser"; + sqlite> DROP TABLE "portal_pendingauthority"; + sqlite> DROP TABLE "portal_pendingslice"; + sqlite> .quit +# ./manage.py migrate portal