Talk:VSOP Machine

VsopMember

The new VSOP Server is here and is ready to take over. Thus the important issue now is a plan for migration:

Migration Plan
The following is a convolution of two plans proposed by John and Cédric:

When
I believe the earliest realistic date for doing this is say Feb 4th provided this is convenient for the Garching helpdesk people we need support from (Mathias). I would have said Feb 1st but that's a Friday so a bad idea for a major migration. This gives time to get everything in place once everyone is back from festive season vacations. Alternatively it might be "cleaner" to migrate as of the start of the new ESO period, but since we have no ESO time but lots of Gemini time, maybe it would be better to migrate on the change of Gemini "period" or whatever they call it there...

How
Here are the tasks to be performed in the order in which they must be performed, with the person responsible for doing it in bold after each task. As tasks are actually done the person responsible will update this page with the date and time at which the task was completed, and then send an email to Thomas, Cédric, Eric & John advising... of course when one person is responsible for several consequetive tasks one email summarising all the tasks will suffice...


 * 1) Collect and make a copy on our own machine of the latest modified code on the Graching Server here and there. This applies to Eric and Cédric. Monobook.js, .css, templates and so on. Eric and/or Cédric :: Cédric claims this is done as of email with Subject 'Re: Access Control working fine: VSOP site switch ready.' dated 23 January 2008 3:50:32 PM
 * 2) Stop access to the Vitacura Wiki with a note saying "we'll be back soon"... Alternatively it would be better to make the entire site read-only, can we do this within the Wiki? Eric and/or Cédric :: John (talk • edits • [mailto:j.pritchard@eso.org?subject=VSOP mail]) 05:00, 5 February 2008 (CLST)
 * 3) Stop ssh access to the Vitacura site: this means all current processes which auto-upload (e.g. data uploads, HARPS?) to that server must be stopped. Easy: delete/rename the authorized_keys file... John :: John (talk • edits • [mailto:j.pritchard@eso.org?subject=VSOP mail]) 05:00, 5 February 2008 (CLST)
 * 4) Dump the Vitacura mysql DB to file. John :: Dump file created on vsop@vsop.sc.eso.org with command: mysqldump -u vsop -p vsop > vsop-`date +%Y-%m-%dT%H:%M:%S`.sql Uploaded to Syphon. Mathias informed by email, Update of Garching Wiki Server scheduled for "this afternoon" according to phone call conversation with Mathias. Note this annotation is NOT inculded in the dump, will have to copied over manually this afternoon John (talk • edits • mail) 05:26, 5 February 2008 (CLST)
 * 5) Deliver it to Mathias who will "upgrade" it to be compatible with the new server's Wiki and then install it overwritng the existing Wiki on the Garching Server. John :: John (talk • edits • [mailto:j.pritchard@eso.org?subject=VSOP mail]) 13:41, 5 February 2008 (CET)
 * 6) rsync other relevant directoires, i.e. the mediawiki directory (to catch any recently submitted images for example)... John images and data done John (talk • edits • [mailto:j.pritchard@eso.org?subject=VSOP mail]) 10:47, 11 February 2008 (CET)
 * 7) Implement the code modifications. Eric and/or Cédric According to email from Cédric dated 11 February 2008 11:16:18 AM with Subject Re: VSOP Server Migration complete
 * 8) Run a robot to control access of every restricted pages as in the previous server. Eric and/or Cédric According to email from Cédric dated 11 February 2008 11:16:18 AM with Subject Re: VSOP Server Migration complete
 * 9) We go online with the new server. John
 * 10) We ask ESO to implement the "simplist" solution such that http://vsop.sc.eso.org is automatically redirected to https://vsop.eso.org. John
 * 11) We are done.

Historical jabber
The VSOP machine is currently hosted at ESO and we have only an ftp access to it. Since we will need to install some tools on it, we need a real access to the machine. According to Thomas Oulevey we are not the only one to use this machine.

Since we will need quite a lot of disk, we may want to use a dedicated machine for VSOP. This would grant us an easy access to it.

We would need an official contact at ESO for this machine, someone working at ESO.

We also have to decide the machine we will need for that.

Let's talk abot it here! --


 * Yo. The machine has been used for deep impact (check the name: vsop.sc.eso.org is an alias to di.sc.eso.org). I don't quite remember how much disk space we have, but it should be like 10 Gb. I think it is a good idea to ask for a real ESO project, but we must know before what it means exactly. Note that we have a big advantage now: daily backup of the pages and the mysql database. -- C édric [[Image:kmail.png|16px]]T alk  07:11, 4 December 2006 (CLST)

I'm in favor of having an ESO machine too, and it would allow us to continue with the current one. For an official ESO project, we must have someone inside the org to act as the machine-PI and main responsible for it. I suggest someone from ESO Chile, although technically John could do it as well. -- Tdall 16:33, 4 December 2006 (CLST)

Subversion instead of CVS
Following email exchange, first from John:
 * From the sound of it I have had a bit more experience than Eric with using and managing a CVS system. Therefore I can offer to either take over from Eric if *HE* would prefer not to be responsible, or else act as a first point of reference in case he has any problems/ questions/concerns.


 * To answer a few points already:


 * As Eric points out, subversion is probably the better way to go, rather than CVS, since it offers a number of advantages, in particular it can be operated over HTTP, unlike CVS where rsh or ssh access would be required, thus it would simplify immensely what would otherwise be the most critical issue for CVS which would be to provide a means of access to everyone in the collaboration both from within and from without the ESO firewall. It will also simplify greatly user Admin; SubVersion users will be private to SubVersion (Like Wiki Users are private to Wiki) rather than using the system users (i.e. Comp Admin won't have to be worried about VSOP needing to create actual users on vsop.sc.eso.org).


 * Subversion is part of the Fedora Core 3 distribution, so should be easy for SysOps in Vitacura to install it. (We can't even build our own version because there is not even a C compiler on vsop.sc.eso.org...) There will still need to be some SysOp configuration, but it should be much less than would be required for CVS...


 * So... let's get started on this, the sooner the better... --

And then reply from Eric:


 * I think that since I am not really experienced in managing this kind of thing it is wiser that you take over from me and that I support you in this task. I am sure that 2 persons in charge of it will not be too much!


 * I will go tomorrow to Vitacura to see Thomas Oulevey to talk about installing subversion in our machine and get a real acces to it. I'll let you know how it goes! --

To which Thomas answers:


 * Subversion is the way to go. Thanks John, for stepping in! Eric, we still need you locally to make sure that some things happen. Remember that we are in the process of installing Carlos' tool for automatic stellar parameters (VASP), and for that we need you locally to take care of the installation. This is one of our top priorities at the moment, because Paper One depends on it. So when you talk to Thomas, make it clear that we need at least one person (you, for now) with full access to the machine.


 * And, to spell out our priorities:
 * Priority 1: install VASP and get it working. Deadline = two weeks!
 * Priority 2: install and configure subversion. Deadline = three months...
 * Please keep this in mind too. -- Tdall 16:29, 2 January 2007 (CLST)

Discussions/Negotiations with ESO-IT
I will talk to Giorgio Fillipi, head of ESO-IT, next week (he is on vacation till then) about a VSOP server. --

Software installations
Thomas Oulevey has installed all the softwares we need. Here is a list of what is already installed:

compilers

 * 1) ifort (the Intel Fortran Compiler) version 9.1,
 * 2) gcc (the Gnu Compiler Compilation) version 3.4.4 ( gcc, g++ and g77 are there also. When we have our own machine in Garching, it will be wise to update gcc to the latest version)

python modules

 * 1) biggles v 1.6.4
 * 2) matplotlib v 0.87
 * 3) pyfits v 1.0.1
 * 4) numpy v 1.0.1
 * 5) Numeric v 24.2

Perl Modules

 * 1) Astro-FITS-CFITSIO
 * 2) Astro-FITS-Headers
 * 3) PDL 2.4.3

Others

 * 1) GNU plotutils v 2.4.1 Eric 09:18, 26 January 2007 (CLST)

Hardware
If we can choose each piece of the server, including the CPU, I would go for AMD ( not only because Intel provides CPUs to Apple ;-) ) since their high end CPUs are way more powerfull than Intel. And if we don't need high end CPUs, AMDs are cheaper. Eric 10:17, 9 February 2007 (CLST)

OS
I would definatively go for Linux, since all our software run under it. Moreover, remote administration is as esay to perform than local administration.
 * Which means what it means exactly. ;-) -- 15:07, 9 February 2007 (CLST)
 * I only meant that I don't know if remote logging is possible if we use less powerful operating systems ;-) Eric 16:22, 9 February 2007 (CLST)

Hardware

 * CPU: Pentium Dual Core???
 * RAM: ???

CPU and RAM requirements depend principally on whether we want to use the machine for Data post-processing or if we do this remotely and then sync the results back to the server. Personally I prefer the latter, and I believe ESO-IT will strongly prefer the latter.


 * HD: On the current VSOP server we use approximately 12Gb. So allowing for continued growth over a year or three 100 or 200 GB should be adequate for a while, unless our usage profile dramatically alters in the near (or far) future.

Software
Here we must distiguish between system software and user software. For the purposes of developing this list of requirements we need only consider the system software required. Anything that can be compiled and/or run by a 'normal' user need not concern us here, except in the sense of insuring that any system sotware required to compile it (gcc for example) is include in the list of requirements. In many cases the distinction between system and user is blurry, e.g. we could certainly maintain our own gcc compiler installation, but we almost certainly don't want to do that. The web server and SubVersion requirements however must be configured (at least to a certain extent) by superuser (and obviously ESO will want to maintain control over this) and therefore acre clearly system


 * Preferred OS: Linux??? Fedora Core??? Scientific Linux???, Possibly we won't have a choice regarding this.
 * Standard comilers: gcc suite, version 4.2 or more recent, which includes, but is not limited to C, C++ and Fortran compilers, ifort, the Intel compiler (We need to check with fortran writing guys what they really want. It seems that the best fortran compiler is not free ( the lahey fortran compiler). They need to let us knkow what they really need.
 * Wiki MediaWiki 1.10 will be soon the latest stable release. Better to go with a fully updated and supported version. 1.6 becomes too old.
 * Wiki Extention: Group Base Access Control to recover the restrict feature not available in MW 1.6+.
 * MediaWiki 1.7 and up need PHP5.+ and MySQL4+
 * Subversion, version 1.4.2 or more recent
 * Perl, version 5.8 or more recent ( latest version is 5.8.8)
 * Perl modules : CPANPLUS ( vital to perform installation of other modules), ASTRO::FITS::CFITSIO, PDL, PGPLOT. Bear in mind, provided we have the appropriate compilers, Perl Modules can always be installed "locally", i.e. for the vsop user, which means no sysadmin intervention is necessary.
 * Python, version 2. (2.5 latest, but any 2.+ would fit).
 * Python modules: pyfits, biggles, numpy. Can these modules be installed "locally" as for Perl, ordo they HAVE to be installed as a sysadmin.
 * SciSoft, version?

Access
Who will access the server, and how:


 * http: The whole world
 * Wiki via http: Collaboration members
 * Subversion via http: Collaboration members
 * ssh: Limited number of collaboration members for the purpose of installation and maintenance of user software. This will almost certainly be handled in the standard ESO ssh access way, namely that we will either have a generic VSOP account providing ssh access to sirius.eso.org and from there ssh access to the VSOP server. Alternatively accounts will be assigned to collaboration members on an individual basis. As John (talk • edits • [mailto:j.pritchard@eso.org?subject=VSOP mail]) 09:31, 9 February 2007 (CLST) will be the one responsible for this it is up to hime to decide, and he decides on individual accounts as required.