Showing posts with label Linux. Show all posts
Showing posts with label Linux. Show all posts

January 12, 2015

How can I change my Cygwin home folder after installation?

Firstly, set a Windows environment variable for HOME that points to your user profile:

  1. Go to Control Panel --> System and Security --> System (or press the [WINDOWS]+PAUSE|BREAK] keys)
  2. Click Advanced system settings (located at the left)
  3. Click Environment Variables (toward the bottom)
  4. In the User Variables area click "New…"
  5. For Variable name enter HOME
  6. For Variable value enter %USERPROFILE%
  7. Click OK in all the open dialog boxes to apply this new setting

Now we are going to update the Cygwin /etc/passwd file with the Windows %HOME% variable we just created. Shell logins and remote logins via ssh will rely on /etc/passwd to tell them the location of the user's $HOME path.

At the Cygwin bash command prompt type the following:

cp /etc/passwd /etc/passwd.bak
mkpasswd -l -p $(cygpath -H)  > /etc/passwd
mkpasswd -d -p $(cygpath -H)  >> /etc/passwd

The -d switch tells mkpasswd to include DOMAIN users as well as LOCAL machine users. This is important if you're using a PC at work where the user information is obtained from a Windows Domain Controller.

Now, do the same for groups by executing the following:

cp /etc/group /etc/group.bak
mkgroup -l > /etc/group
mkgroup -d >> /etc/group

Now, exit Cygwin and start it up again. You should find that your HOME path points to the same location as your Windows User Profile -- i.e. /cygdrive/c/Users/username

©

December 4, 2014

The Quick and Dirty Guide for Parallelizing FLUKA

(Single PC version)

Imagine you got a desktop or laptop PC with 4 or perhaps even 8 CPU cores available, and you want to run the Monte Carlo particle transport program  FLUKA on it using all CPU cores.
The FLUKA execution script rfluka however was designed to run in "serial" mode. That is, if you request to repeat your simulation a lot of times (say, 100) issuing the command rfluka -N0 -M100 example, each process is launched serially, instead of utilizing all available cores on your PC.

A solution can be to use a job queuing system and a scheduler. Here, I'll present one way to do it on a Debian based Linux system. Ubuntu might work just as well, since Ubuntu is very similar to Debian. A feature of the method presented here, is that it can easily be extended to cover several PCs on your network, so you can use the computing power of your colleagues when they do not use their PCs (e.g. at night). However, this post will try to make it very simple, namely set it just on your own PC. In less than 10 minutes you'll have it up and running...

The idea is to use TORQUE in a very minimal configuration. There will be no fuzz with Maui or similar schedulers, we will only use packages we can get from the Debian/Ubuntu software repositories.
In order to be friendly to all the Ubuntu users out there, all commands issued as root are here prefixed with the "sudo" command. As a Debian user you can become root using the "su" command first. 

First install these packages:

$ sudo apt-get install torque-server torque-scheduler $ sudo apt-get install torque-common torque-mom libtorque2 and either
$ sudo apt-get install torque-client or
$ sudo apt-get install torque-client-x11
after installation we need to setup torque properly. I here assume that your PC hostname cannot be resolved by DNS, which is quite common on small local networks. You can test whether your hostname can be resolved by the "host" command. Assuming your PC has the name "kepler", you may get an answer like:

$ host $HOSTNAME Host kepler not found: 3(NXDOMAIN)
this means you may need to edit the /etc/hostsfile, so your PC can associate an IP number with your hostname. Debian like distros may have a propensity to assign the hostname to 127.0.1.1 which will not work with torque. Instead I looked up my IP number (which in my case is pretty static) using /sbin/ifconfig, and edited the/etc/hosts accordingly, using your favourite text editor (emacs, gedit, vi...)
My /etc/hosts file ended up looking like this:

127.0.0.1 localhost #127.0.1.1 kepler.lan kepler 192.168.1.108 kepler
If your hostname of your PC can be resolved, you can ommit the last line, but under all circumstances you must comment out the line starting with 127.0.1.1.

Once this is done, execute the following commands to configure torque:
$ sudo echo $HOSTNAME > /etc/torque/server_name $ sudo echo $HOSTNAME > /var/spool/torque/server_name $ sudo pbs_server -t create $ sudo echo $HOSTNAME np=`grep proc /proc/cpuinfo | wc -l` > /var/spool/torque/server_priv/nodes $ sudo qterm $ sudo pbs_server $ sudo pbs_mom
(Update: If qterm fails, you probably have a problem with your /etc/hosts file. You can still kill the server with $killall -r "pbs_*".)

Now let's  see if things are running as expected:
$ pbsnodes -a kepler state = free np = 4 ntype = cluster status = rectime=1326926041,varattr=,jobs=,state=free,netload=3304768553,gres=,loadave=0.09,ncpus=4,physmem=3988892kb,availmem=6643852kb,totmem=7876584kb,idletime=2518,nusers=2,nsessions=8,sessions=1183 1760 2170 2271 2513 15794 16067 16607,uname=Linux kepler 3.1.0-1-amd64 #1 SMP Tue Jan 10 05:01:58 UTC 2012 x86_64,opsys=linux
and also
$sudo momctl -d 0 -h $HOSTNAME Host: kepler/kepler Version: 2.4.16 PID: 16835 Server[0]: kepler (192.168.1.108:15001) Last Msg From Server: 279 seconds (CLUSTER_ADDRS) Last Msg To Server: 9 seconds HomeDirectory: /var/spool/torque/mom_priv MOM active: 280 seconds LogLevel: 0 (use SIGUSR1/SIGUSR2 to adjust) NOTE: no local jobs detected
Now setup a queue, which here is called "batch".
$ sudo qmgr -c 'create queue batch' $ sudo qmgr -c 'set queue batch queue_type = Execution' $ sudo qmgr -c 'set queue batch resources_default.nodes = 1' $ sudo qmgr -c 'set queue batch resources_default.walltime = 01:00:00' $ sudo qmgr -c 'set queue batch enabled = True' $ sudo qmgr -c 'set queue batch started = True' $ sudo qmgr -c 'set server default_queue = batch' $ sudo qmgr -c 'set server scheduling = True'
[update: you may want to increase walltime to 10:00:00 so jobs dont stop after 1 hour

and start the scheduler:
$ sudo pbs_sched
The rest of the commands can be issued as a normal user (i.e. non-root).

Let's see if all servers are running:
$ ps -e | grep pbs 1286 ? 00:00:00 pbs_mom 1293 ? 00:00:00 pbs_server 2174 ? 00:00:00 pbs_sched
Anything in the queue?
$ qstat $ Nope, it's empty.

Lets try to submit a simple job
echo "sleep 20" | qsub
and within the next 20 seconds you can test, if its in the queue:
$ qstat Job id Name User Time Use S Queue ------------------------- ---------------- --------------- -------- - ----- 0.kepler STDIN bassler 0 R batch

Great, now were ready to rock 'n roll! This is really a minimalistic setup, which just works. For more bells and whistles, check the torque manual.

All we need, is a simple FLUKA job submission script: rtfluka.sh 
#!/bin/bash # # how to use this # change to directory with the files you want to run # and enter: # $ qsub -V -t 0-9 -d . rtfluka.sh # #PBS -N FLUKA_JOB # start="$PBS_ARRAYID" let stop="$start+1" stop_pad=`printf "%03i\n" $stop` # # Init new random number sequence for each calculation. # This may be a poor solution. cp $FLUPRO/random.dat ranexample$stop_pad sed -i '/RANDOMIZE 1.0/c\RANDOMIZE 1.0 '"${RANDOM}"'.0 \' example.inp $FLUPRO/flutil/rfluka -N$start -M$stop example -e flukadpm3
Update: Note that your RANDOMIZE card in your own .inp file must match the sed regular expression above, else you may repeat the exact same simulation over and over again... 

Let's submit 10 jobs:
$ qsub -V -t 0-9 -d . rtfluka.sh
And watch the blinkenlichts.
$ qstat Job id Name User Time Use S Queue ------------------------- ---------------- --------------- -------- - ----- 15-0.kepler FLUKA_JOB-0 bassler 0 R batch 15-1.kepler FLUKA_JOB-1 bassler 0 R batch 15-2.kepler FLUKA_JOB-2 bassler 0 R batch 15-3.kepler FLUKA_JOB-3 bassler 0 R batch 15-4.kepler FLUKA_JOB-4 bassler 0 Q batch 15-5.kepler FLUKA_JOB-5 bassler 0 Q batch 15-6.kepler FLUKA_JOB-6 bassler 0 Q batch 15-7.kepler FLUKA_JOB-7 bassler 0 Q batch 15-8.kepler FLUKA_JOB-8 bassler 0 Q batch 15-9.kepler FLUKA_JOB-9 bassler 0 Q batch
Surely, this can be improved a lot, suggestions are most welcome in the comments below. One problem is for instance, that the random number seed is limited to a 16 bit integer, which only covers a very small fraction of the possible seeds for the RANDOMIZE card.
Update: There is also a very small risk that the same seed occasionally is used twice (or more often). Alternatively one could just add a random number to a starting seed after each run. (Any MC random number experts out there?)

Output data can be processed in regular ways, using flair.  
Alternatively you may use some of the scripts in the auflukatools package, which for instance can do the merging of USRBIN output with a single command. Auflukatools also includes rtfluka.shas well as a CONDOR job submission scriptrcfluka.py, which is better suited for heterogenous clusters.

Finally, here is a job script for SHIELD_HITxxA, (which is even shorter):

#!/bin/bash # # how to use # change to directory you want to run # $ qsub -V -t 0-9 -d . rtshield.sh # #PBS -N SHIELD_JOB shield_exe -N$PBS_ARRAYID
Enjoy!

Totally unrelated: englishrussia.com just posted some nice pics from the Budker institute for Nuclear Physics in Novosibirsk, Russia. Certainly worth visiting, have a look at: 
http://englishrussia.com/2012/01/21/the-budker-institute-of-nuclear-physics/
 :-) Heaps of pioneering accelerator technology was developed there, such as electron cooling, the first collider, lithium lenses (e.g. for capturing antiprotons), and they supplied the conventional magnets for the beam transfer lines to the LHC at CERN. I visited the center many years ago but my pics are not as good. :-/ The German wiki about Budker himself, is also worth reading.

Posted by Niels Bassler at 22:01 

Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest

Labels: BudkerFLUKALinuxMonte Carlo,parallelizationSHIELD-HIT

Location: Aarhus, Denmark

11 comments:

JohnJanuary 23, 2012 at 7:38 AM

Nice work.

Reply

ArminJanuary 23, 2012 at 2:17 PM

This is really fun! Now, I just have to find some more CPUs ...

Reply

Replies

nbasslerJanuary 23, 2012 at 3:30 PM

That will be covered in a future post... :)

Reply

tmarthalFebruary 7, 2012 at 6:44 AM

Have you ever tried to run your FLUKA processes as map tasks in a Hadoop cluster? without knowing anything about it, it would seem that the Hadoop scheduling will take care of all of that for you and the merging/reduce step as well.

Reply

Replies

nbasslerFebruary 7, 2012 at 5:12 PM

Thanks for your comment. No, I have never messed with Hadoop, but I think Roy might have (you reading this, Roy?).

So far I liked the pure torque idea, since all packages are available in the Debian repository. They are pretty good preconfigured, so you don't have to mess with generic configuration scripts and worry how to have a seamless system integration. Updates flow in automatically from Debian, again less trouble for me, since I do not have to get and install new tarballs myself.
Finally, data merging isn't really an issue, due to flair and auflukatools. 

However, if Hadoop installation and configuration is more simple than what I mentioned in this post, or simplify other parts, let me know...

RoyFebruary 8, 2012 at 6:43 AM

We messed around some with Fluka + Hadoop. The general mapreduce work flow doesn't seem ideal for this, but is certainly possible. Our collaborators used a tree-based overlay network (TBON), which is like a generalized mapreduce to run Fluka jobs for us on our local cluster. Of course this was more of a proof of principle test.

We messed around with Hadoop some, but we ended up using our in house framework to run our cloud calculations. Hadoop is Java-based, which I'm not a fan of. I did briefly look at some other mapreduce frameworks, such as dumbo (python-based).

Overall doing Monte Carlo with mapreduce sort of feels like forcing a square peg into a round hole. Monte Carlo already consists of n independent runs (n being an arbitrary number), making the whole mapping / partitioning part of mapreduce frameworks an awkward fit.

Reply

RoyFebruary 8, 2012 at 6:46 AM

Excellent as always Neils!

Reply

Replies

nbasslerFebruary 8, 2012 at 1:26 PM

Thanks, Ryo!

Reply

VasilisMarch 27, 2012 at 9:41 PM

You can download the qfluka.sh form the flair website (download section). It works for torque you simply have to put the correct queue name. Add it to the flair queue system. "Tools->Preferences->Submit->right-click->Add"
and then simply "Spawn" your input in the "Run" Frame to as many CPU's you have select the submit script and click run.
Flair will automatically do everything for you as you describe in your page. Put a unique random number and spawn many processes, as well collect and process your results.

Reply

Replies

nbasslerApril 21, 2012 at 10:21 PM

Very good, thanks for pointing this out, Vasilis!

Reply

Xijun WangOctober 22, 2013 at 12:48 AM

Upon doing "echo 'sleep 20' | qsub", I kept getting error like:
$ qsub: Bad UID for job execution MSG=ruserok failed validating xxxx/xxxx from xxxx-XXXX

Google helped me to solve the question by adding the following qmgr:
$ sudo qmgr -c 'set server allow_node_submit = True'

©

November 4, 2014

How to set up default group permission in linux folder

Q:

I have website folder called

/home/john/public_html

Now i want that what ever files are copied to that public_html or new files created inside that folder then those files should have default read /write permission by john , no matter who is the owner of that file.

Also i will be copying /creating files /directories by FTP , will that system work there as well or file with ftp won't have group read /write permission

i mean will umask work only files created / copied within shell or even outside shell aswell

A:

I'm not exactly sure what you're trying to accomplish, but it looks like it could be solved with group permissions and a setgid bit on the directory.

chown john.john-contrib /home/john/public_html
chmod 2775 /home/john/public_html

Then when people create files, they'll be owned by the group 'john-contrib'. Everyone needs to be a member of john-contrib to put files in John's directory though.

If you really want everyone to write into John's directory all the time without any security controls... then chmod 2777 would work, but that's a little insane application of the /home filesystem.
©

September 3, 2014

Appendix L: TORQUE Quick Start Guide

L.1 Initial Installation

  • Download the TORQUE distribution file from http://clusterresources.com/downloads/torque
  • Extract and build the distribution on the machine that will act as the "TORQUE server" - the machine that will monitor and control all compute nodes by running the pbs_server daemon. See the example below:
    > tar -xzvf torque.tar.gz
    > cd torque
    > ./configure
    > make
    > make install
    
    Note OSX 10.4 users need to change the #define __TDARWIN in src/include/pbs_config.h to #define __TDARWIN_8.
  • Note After installation, verify you have PATH environment variables configured for /usr/local/bin/ and /usr/local/sbin/. Client commands are installed to /usr/local/bin and server binaries are installed to /usr/local/sbin.<
    Note In this document TORQUE_HOME corresponds to where TORQUE stores its configuration files. The default is /var/spool/torque.

L.2 Initialize/Configure TORQUE on the Server (pbs_server)

  • Once installation on the TORQUE server is complete, configure the pbs_server daemon by executing the command torque.setup <USER> found packaged with the distribution source code, where <USER> is a username that will act as the TORQUE admin. This script will set up a basic batch queue to get you started. If you experience problems, make sure that the most recent TORQUE executables are being executed, or that the executables are in your current PATH.
  • If doing this step manually, be certain to run the command 'pbs_server -t create' to create the new batch database. If this step is not taken, the pbs_server daemon will be unable to start.
  • Proper server configuration can be verified by following the steps listed in Section 1.4 Testing

L.3 Install TORQUE on the Compute Nodes

To configure a compute node do the following on each machine (see page 19, Section 3.2.1 of PBS Administrator's Manual for full details):
  • Create the self-extracting, distributable packages with make packages (See the INSTALL file for additional options and features of the distributable packages) and use the parallel shell command from your cluster management suite to copy and execute the package on all nodes (ie: xCAT users might do prcp torque-package-linux-i686.sh main:/tmp/; psh main /tmp/torque-package-linux-i686.sh --install. Optionally, distribute and install the clients package.

L.4 Configure TORQUE on the Compute Nodes

  • For each compute host, the MOM daemon must be configured to trust the pbs_server daemon. In TORQUE 2.0.0p4 and earlier, this is done by creating the TORQUE_HOME/mom_priv/config file and setting the $pbsserver parameter. In TORQUE 2.0.0p5 and later, this can also be done by creating the TORQUE_HOME/server_name file and placing the server hostname inside.
  • Additional config parameters may be added to TORQUE_HOME/mom_priv/config (See the MOM Config page for details.)

L.5 Configure Data Management on the Compute Nodes

Data management allows jobs' data to be staged in/out or to and from the server and compute nodes.
  • For shared filesystems (i.e., NFS, DFS, AFS, etc.) use the $usecp parameter in the mom_priv/config files to specify how to map a user's home directory.
    (Example: $usecp gridmaster.tmx.com:/home /home)
  • For local, non-shared filesystems, rcp or scp must be configured to allow direct copy without prompting for passwords (key authentication, etc.)

L.6 Update TORQUE Server Configuration

  • On the TORQUE server, append the list of newly configured compute nodes to the TORQUE_HOME/server_priv/nodes file:
    server_priv/nodes
    computenode001.cluster.org
    computenode002.cluster.org
    computenode003.cluster.org
    

L.7 Start the pbs_mom Daemons on Compute Nodes

  • Next start the pbs_mom daemon on each compute node by running the pbs_mom executable.

L.8 Verifying Correct TORQUE Installation

The pbs_server daemon was started on the TORQUE server when the torque.setup file was executed or when it was manually configured. It must now be restarted so it can reload the updated configuration changes.
# shutdown server
> qterm # shutdown server

# start server
> pbs_server

# verify all queues are properly configured
> qstat -q

# view additional server configuration
>  qmgr -c 'p s'

# verify all nodes are correctly reporting
> pbsnodes -a 

# submit a basic job
> echo "sleep 30" | qsub

# verify jobs display
> qstat
At this point, the job will not start because there is no scheduler running. The scheduler is enabled in the next step below.

L.9 Enabling the Scheduler

Selecting the cluster scheduler is an important decision and significantly affects cluster utilization, responsiveness, availability, and intelligence. The default TORQUE scheduler, pbs_sched, is very basic and will provide poor utilization of your cluster's resources. Other options, such as Maui Scheduler or Moab Workload Manager are highly recommended. If using Maui/Moab, refer to the Moab-PBS Integration Guide. If using pbs_sched, start this daemon now.
Note If you are installing ClusterSuite, TORQUE and Moab were configured at installation for interoperability and no further action is required.

L.10 Startup/Shutdown Service Script for TORQUE/Moab (OPTIONAL)

Optional startup/shutdown service scripts are provided as an example of how to run TORQUE as an OS service that starts at bootup. The scripts are located in the contrib/init.d/ directory of the TORQUE tarball you downloaded. In order to use the script you must:
  • Determine which init.d script suits your platform the best.
  • Modify the script to point to TORQUE's install location. This should only be necessary if you used a non-default install location for TORQUE (by using the --prefix option of ./configure).
  • Place the script in the /etc/init.d/ directory.
  • Use a tool like chkconfig to activate the start-up scripts or make symbolic links (S99moab and K15moab, for example) in desired runtimes (/etc/rc.d/rc3.d/ on Redhat, etc.).

See Also:

©

Torque configuration notes on Ububtu 12.04

Client:
$ sudo apt-get install torque-common torque-client torque-mom

$ sudo nano /var/spool/torque/server_name:
server_name

$ sudo nano /var/spool/torque/mom_priv/config
$pbsserver      server_name          # note: this is the hostname of the headnode

start pbs_mom:
$ pbs_mom

or reboot


Server:
$sudo nano /var/spool/torque/server_priv/nodes
node_name np=4

Torque configuration on Archlinux


TORQUE is an open source resource manager providing control over batch jobs and distributed compute nodes. Basically, one can setup a home or small office Linux cluster and queue jobs with this software. A cluster consists of one head node and many compute nodes. The head node runs the torque-server daemon and the compute nodes run the torque-client daemon. The head node also runs a scheduler daemon.

Installation

Note: Although TORQUE is a very powerful queuing system, if the goal of the cluster is solely to increase compilation throughput, distcc is a much easier and elegant solution.
Install the torque package found in the AUR.

Must haves

/etc/hosts

Make sure that /etc/hosts on all of the boxes in the cluster contains the hostnames of every PC in the cluster. Example, cluster consists of 3 PCs, mars, phobos, and deimos.
192.168.0.20   mars
192.168.0.21   phobos
192.168.0.22   deimos

Firewall configuration (if installed)

Be sure to open TCP for all machines using TORQUE.
The pbs_server (server) and pbs_mom (client) by default use TCP and UDP ports 15001-15004. pbs_mom (client) also uses UDP ports 1023 and below if privileged ports are configured (the default).

NFS

Technically, one does not need to use NFS but doing so simplifies the whole process. An NFS share either on the server or another machine is highly recommended to simplify the process of sharing common build disk space.

Setup

Server (head node) configuration

Follow these steps on the head node/scheduler.
Edit /var/spool/torque/server_name to name the head node. It is recommended to match the hostname in /etc/hostname for simplicity's sake.
Create and configure the torque server:
# pbs_server -t create
PBS_Server localhost.localdomain: Create mode and server database exists,
do you wish to continue y/(n)?y
A minimal set of options are provided here. Adjust the first line substituting "mars" with the hostname entered in /var/spool/torque/server_name:
qmgr -c "set server acl_hosts = mars"
qmgr -c "set server scheduling=true"
qmgr -c "create queue batch queue_type=execution"
qmgr -c "set queue batch started=true"
qmgr -c "set queue batch enabled=true"
qmgr -c "set queue batch resources_default.nodes=1"
qmgr -c "set queue batch resources_default.walltime=3600"
qmgr -c "set server default_queue=batch"
It may be of interest to keep finished jobs in the queue for a period of time.
qmgr -c "set server keep_completed = 86400"
Here, 86400 sec = 24 h after which point, the job will be auto removed from the queue. One can see the full log of jobs removed from the queue with the -f switch on qstat:
qstat -f
Verify the server config with this command:
# qmgr -c 'p s'
Edit /var/spool/torque/server_priv/nodes adding all compute nodes. Again, it is recommended to match the hostname(s) of the machines on the LAN. The syntax is HOSTNAME np=x gpus=y properties
  • HOSTNAME=the hostname of the machine
  • np=number of processors
  • gpus=number of gpus
  • properties=comments
Only the hostname is required, all other fields are optional.
Example:
mars np=4
phobos np=2
deimos np=2
Note:
  • One can run both the server and client on the same box.
  • Re-running pbs_server -t create may delete this nodes file.
Restart the server and the new options are sourced.

Client (compute node) configuration

Follow these steps on each compute node in the cluster.
Note: If running both the server and client on the same box, be sure to complete these steps as well for that machine as well as other pure clients on the cluster.
Edit /var/spool/torque/mom_priv/config to contain some basic info identifying the server:
$pbsserver      mars          # note: this is the hostname of the headnode
$logevent       255           # bitmap of which events to log

Restart the server and client(s)

That should be it. Restart the server and the client: torque-server torque-node.

Verifying cluster status

To check the status of the cluster, issue the following:
$ pbsnodes -a
Each node if up should indicate that it is ready to receive jobs echoing a state of free. If a node is not working, it will report a state of down.
Example output:
mars
     state = free
     np = 4
     ntype = cluster
     status = rectime=1308479899,varattr=,jobs=0.localhost.localdomain,state=free,netload=1638547057,
gres=,loadave=2.69,ncpus=4,physmem=8195892kb,availmem=7172508kb,totmem=8195892kb,
idletime=24772,nusers=1,nsessions=5,sessions=1333 1349 1353 1388 9095,
uname=Linux mars 2.6.39-ck #1 SMP PREEMPT Sat Jun 18 14:19:01 EDT 2011 x86_64,opsys=linux
     mom_service_port = 15002
     mom_manager_port = 15003
     gpus = 2

phobos
     state = free
     np = 2
     ntype = cluster
     status = rectime=1308479933,varattr=,jobs=,state=free,netload=1085755815,
gres=,loadave=2.84,ncpus=2,physmem=4019704kb,availmem=5753552kb,totmem=6116852kb,
idletime=7324,nusers=2,nsessions=6,sessions=1565 1562 1691 1716 1737 1851,
uname=Linux phobos 2.6.37-ck #1 SMP PREEMPT Sun Apr 3 17:16:35 EDT 2011 x86_64,opsys=linux
     mom_service_port = 15002
     mom_manager_port = 15003
     gpus = 1

deimos
     state = free
     np = 2
     ntype = cluster
     status = rectime=1308479890,varattr=,jobs=2.localhost.localdomain,state=free,netload=527239670,
gres=,loadave=0.52,ncpus=2,physmem=4057808kb,availmem=3955624kb,totmem=4057808kb,
idletime=644,nusers=1,nsessions=1,sessions=865,
uname=Linux deimos 2.6.39-ck #1 SMP PREEMPT Sat Jun 11 12:36:21 EDT 2011 x86_64,opsys=linux
     mom_service_port = 15002
     mom_manager_port = 15003
     gpus = 1

Queuing jobs

Queuing to the cluster is accomplished via the qsub command.
A trivial test is to simply run sleep:
$ echo "sleep 30" | qsub
Check the status of the queue via the qstat command described below. At this point, the work will have a status of "Q" which means queued. To start it, run the scheduler:
# pbs_sched
One can modify the torque-server systemd daemon to activate pbs_sched at boot.
Another usage of qsub is to name a job and queue a script:
$ qsub -N x264 /home/facade/bin/x264_HQ.sh
Note: STDOUT and STDERR for a queued job will be logged by default in the form text files corresponding to the respective outputs pid.o and pid.e and will be written to the path from which the qsub command was issued.
Another example can use a wrapper script to make and queue work en mass automatically.

Checking job status

qstat is used to check work status.
$ qstat
Job id                    Name             User            Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
13.localhost               generic-i686.pbs facade         00:05:06 R batch          
14.localhost               atom-i686.pbs    facade         00:03:09 R batch          
15.localhost               core2-i686.pbs   facade         00:01:02 R batch          
16.localhost               k7-i686.pbs      facade                0 Q batch          
17.localhost               k8-i686.pbs      facade                0 Q batch          
18.localhost               k10-i686.pbs     facade                0 Q batch          
19.localhost               p4-i686.pbs      facade                0 Q batch          
20.localhost               pentm-i686.pbs   facade                0 Q batch          
21.localhost               ...ic-x86_64.pbs facade                0 Q batch          
22.localhost               atom-x86_64.pbs  facade                0 Q batch          
23.localhost               core2-x86_64.pbs facade                0 Q batch          
24.localhost               k8-x86_64.pbs    facade                0 Q batch          
25.localhost               k10-x86_64.pbs   facade                0 Q batch          
Append the -n switch to see which nodes are doing which jobs.
$ qstat -n
localhost.localdomain:
405.localhost.lo     facade  batch    i686-generic       3035     1   0    --  01:00 C 00:12
   mars/3+mars/2+mars/1+mars/0
406.localhost.lo     facade  batch    i686-atom          5768     1   0    --  01:00 C 00:46
   phobos/1+phobos/0
407.localhost.lo     facade  batch    i686-core2        22941     1   0    --  01:00 C 00:12
   mars/3+mars/2+mars/1+mars/0
408.localhost.lo     facade  batch    i686-k7           10152     1   0    --  01:00 C 00:12
   mars/3+mars/2+mars/1+mars/0
409.localhost.lo     facade  batch    i686-k8           29657     1   0    --  01:00 C 00:12
   mars/3+mars/2+mars/1+mars/0
410.localhost.lo     facade  batch    i686-k10          16838     1   0    --  01:00 C 00:12
   mars/3+mars/2+mars/1+mars/0
411.localhost.lo     facade  batch    i686-p4           25340     1   0    --  01:00 C 00:46
   deimos/1+deimos/0
412.localhost.lo     facade  batch    i686-pentm        12544     1   0    --  01:00 R 00:20
   phobos/1+phobos/0
413.localhost.lo     facade  batch    x86_64-generic     4024     1   0    --  01:00 C 00:13
   mars/3+mars/2+mars/1+mars/0
414.localhost.lo     facade  batch    x86_64-atom       19330     1   0    --  01:00 C 00:13
   mars/3+mars/2+mars/1+mars/0
415.localhost.lo     facade  batch    x86_64-core2       2146     1   0    --  01:00 C 00:13
   mars/3+mars/2+mars/1+mars/0
416.localhost.lo     facade  batch    x86_64-k8         17234     1   0    --  01:00 R 00:11
   mars/3+mars/2+mars/1+mars/0
417.localhost.lo     facade  batch    x86_64-k10          --      1   0    --  01:00 Q   -- 
    -- 


©

bash script, erase previous line?

Q: In lots of Linux programs, like curl, wget, and anything with a progress meter, they have the bottom line constantly update, every certain amount of time. How do I do that in a bash script?


A:

{
for pc in $(seq 1 100); do
echo -ne "$pc%\033[0K\r"
usleep 100000
done
echo
}

The "\033[0K" will delete to the end of the line - in case your progress line gets shorter at some point, although this may not be necessary for your purposes.

The "\r" will move the cursor to the beginning of the current line

The -n on echo will prevent the cursor advancing to the next line
©

August 28, 2014

Autojump in 12.04 doesn't work

I installed with apt-get, checked out the man page and added . /usr/share/autojump/autojump.sh to my .bashrc, like it says. When I cd around the filesystem, nothing gets added to ~/.local/share/autojump. I then tried adding . /usr/share/autojump/autojump.bash, but that didn't work either.

---

You'll probably want to add the following to your .bashrc:

source /usr/share/autojump/autojump.bash

©

How To Set Up an NFS Mount on Ubuntu 12.04

Author: Etel Sverdlov Published: Sep 17, 2012 Updated: Jun 11, 2014

About NFS (Network File System) Mounts

NFS mounts work to share a directory between several virtual servers. This has the advantage of saving disk space, as the home directory is only kept on one virtual private server, and others can connect to it over the network. When setting up mounts, NFS is most effective for permanent fixtures that should always be accessible.

Setup

An NFS mount is set up between at least two virtual servers. The machine hosting the shared network is called the server, while the ones that connect to it are called ‘clients’.
This tutorial requires 2 servers: one acting as the server and one as the client. We will set up the server machine first, followed by the client. The following IP addresses will refer to each one:
Master: 12.34.56.789
Client: 12.33.44.555
The system should be set up as root. You can access the root user by typing
sudo su-

Setting Up the NFS Server

Step One—Download the Required Software

Start off by using apt-get to install the nfs programs.
apt-get install nfs-kernel-server portmap

Step Two—Export the Shared Directory

The next step is to decide which directory we want to share with the client server. The chosen directory should then be added to the /etc/exports file, which specifies both the directory to be shared and the details of how it is shared.
Suppose we wanted to share two directories: /home and /var/nfs.
Because the /var/nfs/ does not exist, we need to do two things before we can export it.
First, we need to create the directory itself:
mkdir /var/nfs/
Second, we should change the ownership of the directory to the user, nobody and the group, no group. These represent the default user through which clients can access a directory shared through NFS.
Go ahead and chown the directory:
chown nobody:nogroup /var/nfs
After completing those steps, it’s time to export the directories to the other VPS:
nano /etc/exports
Add the following lines to the bottom of the file, sharing both directories with the client:
/home           12.33.44.555(rw,sync,no_root_squash,no_subtree_check)
/var/nfs        12.33.44.555(rw,sync,no_subtree_check)
These settings accomplish several tasks:
  • rw: This option allows the client server to both read and write within the shared directory
  • sync: Sync confirms requests to the shared directory only once the changes have been committed.
  • no_subtree_check: This option prevents the subtree checking. When a shared directory is the subdirectory of a larger filesystem, nfs performs scans of every directory above it, in order to verify its permissions and details. Disabling the subtree check may increase the reliability of NFS, but reduce security.
  • no_root_squash: This phrase allows root to connect to the designated directory
Once you have entered in the settings for each directory, run the following command to export them:
exportfs -a

Setting Up the NFS Client

Step One—Download the Required Software

Start off by using apt-get to install the nfs programs.
apt-get install nfs-common portmap

Step Two—Mount the Directories

Once the programs have been downloaded to the the client server, create the directories that will contain the NFS shared files
mkdir -p /mnt/nfs/home
mkdir -p /mnt/nfs/var/nfs
Then go ahead and mount them
mount 12.34.56.789:/home /mnt/nfs/home
mount 12.34.56.789:/var/nfs /mnt/nfs/var/nfs
You can use the df -h command to check that the directories have been mounted. You will see them last on the list.
df -h
Filesystem             Size  Used Avail Use% Mounted on
/dev/sda                20G  948M   19G   5% /
udev                   119M  4.0K  119M   1% /dev
tmpfs                   49M  208K   49M   1% /run
none                   5.0M     0  5.0M   0% /run/lock
none                   122M     0  122M   0% /run/shm
12.34.56.789:/home      20G  948M   19G   5% /mnt/nfs/home
12.34.56.789:/var/nfs   20G  948M   19G   5% /mnt/nfs/var/nfs
Additionally, use the mount command to see the entire list of mounted file systems.
mount
Your list should look something like this:
/dev/sda on / type ext4 (rw,errors=remount-ro,barrier=0) [DOROOT]
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
rpc_pipefs on /run/rpc_pipefs type rpc_pipefs (rw)
12.34.56.789:/home on /mnt/nfs/home type nfs (rw,vers=4,addr= 12.34.56.789,clientaddr=12.33.44.555)
12.34.56.789:/var/nfs on /mnt/nfs/var/nfs type nfs (rw,vers=4,addr=12.34.56.78,clientaddr=12.33.44.555)

Testing the NFS Mount

Once you have successfully mounted your NFS directories, you can test that they work by creating files on the Client and checking their availability on the Server.
Create a file in each directory to try it out:
touch /mnt/nfs/home/example /mnt/nfs/var/nfs/example
You should then be able to find the files on the Server in the /home and /var/nfs directories.
ls /home
ls /var/nfs/
You can ensure that the mount is always active by adding the directories to the fstab file on the client. This will ensure that the mounts start up after the server reboots.
nano /etc/fstab
12.34.56.789:/home  /mnt/nfs/home   nfs      auto,noatime,nolock,bg,nfsvers=3,intr,tcp,actimeo=1800 0 0
12.34.56.789:/var/nfs  /mnt/nfs/var/nfs   nfs     auto,noatime,nolock,bg,nfsvers=3,intr,tcp,actimeo=1800 0 0
You can learn more about the fstab options by typing in:
man nfs
Any subsequent restarts will include the NFS mount—although the mount may take a minute to load after the reboot
You can check the mounted directories with the two earlier commands:
df -h
mount

Removing the NFS Mount

Should you decide to remove a directory, you can unmount it using the umount command:
cd
sudo umount /directory name
You can see that the mounts were removed by then looking at the filesystem again.
df -h
You should find your selected mounted directory gone.
©

How do I disable X at boot time so that the system boots in text mode?

For Ubuntu 11.10 and higher
Edit /etc/default/grub with your favorite editor,
sudo nano /etc/default/grub

Find out this line:
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"

Change it to:
GRUB_CMDLINE_LINUX_DEFAULT="text"

Update Grub:
sudo update-grub

No need to remove / disable lightdm upstart conf, it already does that for you.
lightdm.conf
# Check kernel command-line for inhibitors, unless we are being called
        # manually
        for ARG in $(cat /proc/cmdline); do
            if [ "$ARG" = "text" ]; then
                plymouth quit || :
                stop
                exit 0
            fi
        done

You will still be able to use X by typing startx after you logged in.
©

August 20, 2014

Installing R in Ubuntu

Universe

The current version of R available when a new version of Ubuntu is released is always available in the universe repository. To install R:

sudo apt-get install r-base

R package installation in ubuntu
install.packages("fitdistrplus")

©

July 21, 2014

Migrating home directory into its own dedicated partition

Overview

This guide offers detailed instructions for migrating your home directory into its own dedicated partition. Setting up /home on a separate partition is beneficial because your settings, files, and desktop will be maintained if you upgrade, (re)install Ubuntu or another distro. This works because /home has a subdirectory for each user's settings and files which contain all the data & settings of that user. Telling Ubuntu to use an existing home partition can be done by selecting "Manual Partitioning" during the installation of Ubuntu and specifying that you want your home partitions mount point to be /home, ensure you mark your /home partition not be formatted in the process. You should also make sure the usernames you enter for accounts during installation match usernames that existed in a previous installation.

This guide will follow these 6 basic steps:

Setup your new partition
Backup and edit your fstab to mount the new partition as /media/home (just for the time being) and reboot.
Use rsync to migrate all data from /home into /media/home.
Edit fstab again so the new partition mounts as /home instead of /media/home but not reboot just yet.
Move /home to /old_home and reboot
Delete /old_home.

The guide is written in such a way so that at any point in time if there is a system failure, power outage or random restart that it will not have a negative impact on the system and SHOULD safeguard against the possibility of the user accidentally deleting their home directory in the process.

Creating a new partition

Setting up /home on a separate partition is beneficial because your settings, files, and desktop will be maintained if you upgrade, (re)install Ubuntu or another distro. This works because /home has a subdirectory for each user's settings and files which contain all the data & settings of that user. Also, fresh installs for linux typically like to wipe whatever partition they are being installed to so either the data & settings need to be backed-up elsewhere or else avoid the fuss each time by having /home on a different partition.

Setup Partitions

This is beyond the scope of this page. Try here if you need help. Memorize or write down the location of the partition, something like /sda3. When you do create a new partition it is highly suggested that you create an ext3 or ext4 partition to house your new home directory.

Find the uuid of the Partition

The uuid (Universally Unique Identifier) reference for all partitions can be found by opening a command-line to type the following:

sudo blkid

Alternatively, for some older releases of Ubuntu the "blkid" command might not work so this could be used instead

sudo vol_id -u

for example

sudo vol_id -u /dev/sda3

Now you just need to take note (copy&paste into a text-file) the uuid of the partition that you have set-up ready to be the new /home partition.

Setup Fstab

Your fstab is a file used to tell Ubuntu what partitions to mount at boot. The following commands will duplicate your current fstab, append the year-month-day to the end of the file name, compare the two files and open the original for editing.

1. Duplicate your fstab file:

sudo cp /etc/fstab /etc/fstab.$(date +%Y-%m-%d)

2. Compare the two files to confirm the backup matches the original:

cmp /etc/fstab /etc/fstab.$(date +%Y-%m-%d)

3. Open the original fstab in a text editor:

gksu gedit /etc/fstab

and add these lines into it

# (identifier) (location, eg sda5) (format, eg ext3 or ext4) (some settings)
UUID=???????? /media/home ext3 defaults 0 2

and replace the "????????" with the UUID number of the intended /home partition.

NOTE: In the above example, the specified partition in the new text is an ext3, but if yours is an ext4 partition, you should change the part above that says "ext3" to say "ext4", in addition to replacing the ???'s with the correct UUID. Also note that if you are using Kubuntu, Xubuntu or Lubuntu you may need to replace "gedit" with "kate", "mousepad" or "leafpad", respectively. They are text editors included with those distributions.

4. Save and Close the fstab file, then type the following command:

sudo mkdir /media/home

This command will create a new directory, later used for temporarily mounting the new partition. At the end of the procedure this directory can be removed.

Now you can restart your machine or instead of rebooting you might prefer to just re-load the updated fstab

sudo mount -a

Either should have mounted the new partition as /media/home. We will edit the fstab again later so this arrangement of the partition is only temporary.

Copy /home to the New Partition

Next we will copy all files, directories and sub-directories from your current /home directory into the new partition:

sudo rsync -aXS --exclude='/*/.gvfs' /home/. /media/home/.

The --exclude='/*/.gvfs' prevents rsync from complaining about not being able to copy .gvfs, but I believe it is optional. Even if rsync complains, it will copy everything else anyway. (See here for discussion on this)

Check Copying Worked

You should now have two duplicate copies of all the data within your home directory; the original being located in /home and the new duplicate located in /media/home. You should confirm all files and directories copied over successfully. One way to do this is by using the diff command:

sudo diff -r /home /media/home

The only difference that should exist is the excluded /.gvfs directory mentioned above. If you are doing this from a LiveCd or to an exisitng partition that already has stuff on it then you may find other differences but hopefully it should be obvious which diffs you can ignore.

Preparing fstab for the switch

We now need to modify the fstab again to point to the new partition and mount it as /home. So again on a command-line

gksu gedit /etc/fstab

and now edit the lines you added earlier, changing the "/media/home" part to simply say "/home" so that it looks like this:

# (identifier) (location, eg sda5) (format, eg ext3 or ext4) (some settings)
UUID=???????? /home ext3 defaults 0 2

Then, press Save, close the file but don't reboot just yet.

Moving /home into /old_home

Backing up your old home, just in case things have not gone completely smoothly, is best done right now. Here is how:

As long as you have not rebooted yet, you will still see 2 copies of your /home directory; the new one on the new partition (currently mounted as /media/home) and the old one still in the same partition it was always in (currently mounted as /home). We need to move the contents of the old home directory out of the way and create an empty "placeholder" directory to act as a mount point for our new partition.

Type the following string of commands in to do all this at once:

cd / && sudo mv /home /old_home && sudo mkdir /home

By default, when you open a terminal window it places you within your home directory. Typing cd / takes us to the root directory and out of home so we can then use the sudo mv command to essentially rename /home into /old_home, and finally create a new, empty /home placeholder.

With your fstab now edited to mount your new partition to our /home placeholder and the original /home now called /old_home, reboot your computer. Your new partition will mount as /home and everything should look exactly the same as it did before you started.

Instead of rebooting you might prefer to just re-load the updated fstab

sudo mount -a

There is no real need to reboot at all!

Deleting the old Home

You can delete your old home directory with:

cd /
sudo rm -r /old_home

Be careful with the above command as mistyping it could result in the deletion of other files and directories.
Technical Notes and Resources

Rsync was chosen over cp and find|cpio because it seemed to maintain permissions.

http://ubuntu.wordpress.com/2006/01/29/move-home-to-its-own-partition/

http://ubuntuforums.org/showthread.php?t=46866

Different filesystems on the same disk

If you're moving from Windows and your new home partition is going to be an old ntfs partition (the D: disk) while you convert the C: disk to a journaling partition where you install Linux, this won't work, there will be a huge load on the processor. You should convert the two partitions to ext3 or ext4 or keep both partitions as ntfs (I haven't checked this last option). But working with two different filesystems on the same drive simultaneously doesn't seem to be a good option.
©

X11 Connection Rejected Because of Wrong Authentication

by Nix Craft on September 17, 2008 · 34 comments· LAST UPDATED September 17, 2008

in BASH Shell, Debian / Ubuntu, Linux

Q. I'm trying to login to my remote Ubuntu Linux server from Mac OS X desktop using following command:
ssh -X user@vpn.officeserver.example.com xeyes

But I'm getting an error that read as follows:

X11 connection rejected because of wrong authentication.

How do I fix this error?

A. This error can be caused by various factors. Try following solutions:
Make sure you are not running out of disk space

Run df and make sure you have sufficient disk space:
$ df -H
If you are low on disk space remove unnecessary files from your system.
Make sure ~/.Xauthority owned by you

Run following command to find ownweship:
$ ls -l ~/.Xauthority
Run chown and chmod to fix permission problems
$ chown user:group ~/.Xauthority
$ chmod 0600 ~/.Xauthority
Replace user:group with your actual username and groupname.
Make sure X11 SSHD Forwarding Enabled

Make sure following line exists in sshd_config file:
$ grep X11Forwarding /etc/ssh/sshd_config
Sample output:

X11Forwarding yes

If X11 disabled add following line to sshd_cofing and restart ssh server:
X11Forwarding yes
Make sure X11 client forwarding enabled

Make sure your local ssh_config has following lines:
Host *
ForwardX11 yes

Finally, login to remote server and run X11 as follows from your Mac OS X or Linux desktop system:
ssh -X user@remote.box.example.com xeyes

©

December 6, 2013

How To Put a LiveCD/DVD To USB

Scientific Linux Forum.org

In this how to I will describe 4 methods that I use to put a livecd/dvd to USB. All of these methods will delete/destroy any data that is on your USB stick!

1. Using dd (IMMO this is the simplest, most reliable and my favorite)
2. By mounting a cd/dvd in a loop and copying the contents and installing grub
3. Using livecd-iso-to-disk (This method also works for install DVD's)
4. Using liveusb-creator

The USB stick I will be using throughout this how to has a device location of /dev/sdc . It is very important that you determine the correct location of your USB stick and change /dev/sdc to the device location of your USB stick in each of the commands in this how to. Do not just copy past the commands in this how to as that could result in the wiping/overwriting/erasing of your hard drive/s. YOU HAVE BEEN WARNED! rolleyes.gif

HOW TO DETERMINE THE DEVICE LOCATION OF YOUR USB STICK:
There are several ways to determine the device location of your USB stick. I will describe a few here. Use the method that you are comfortable with:

To determine the location of your USB stick from the command line use one of the following:
The lsblk command returns a easy to read summery of the installed drives, there size, there device location and there mount locations if mounted, etc.
CODE
lsblk


Fdisk is also able to list lots of disk info but requires root privileges:
CODE
su -c 'fdisk -l'


The command blkid will list disk info along with the UUID number.
CODE
blkid


To determine the location of your USB stick using the disk utility palimpsest:
Go Applications >> System Tools >> Disk Utility On the left hand side of the disk utility window there is a list of storage devices, locate the one that represents your USB stick and click on it. This will display the info for your USB stick on the right hand side of the disk utility window. The device location of the USB stick is displayed on the top right hand side of the disk utility window. Look for the line resembling “ Device: /dev/sdc “ . From this computer the USB device is located at /dev/sdc. Yours may be different and you will have to adjust the commands accordingly by replacing /dev/sdc with the location of your USB stick as shown by the info following the line “Device: in the disk utility window. Failing to do so may result in you wiping/overwriting/erasing your hard drive/s.



1. THE DD METHOD:
This method works on just about all the available livecd/dvds. I have found this method to be the most reliable method. I have used this method with many different distros including SL, Debian, Fedora, Centos, opensuse, Ubuntu ect.

Insert the USB stick,
determine the device location using one of the above methods
Unmount any mounted partitions that are mounted from the USB stick. (This can be achieved by right mouse clicking on the mounted partition/s on your desktop and selecting umount or opening the disk utility, clicking on your usb stick on the left hand side. Then click on the mounted partition on the right hand side and then click the unmount button. Or use the umount command from the command line)

The nest step is to wipe the USB stick with the following command (make sure to change /dev/sdc to your usb stick's device location for all of the following commands):
CODE
su -
dd if=/dev/zero of=/dev/sdc bs=512 count=1


Now to put the SL-62-i383-2012-02-16-LiveCD.iso to the usb stick. (Change path-to-iso to your real path to the iso file. ie. Something like /home/sluser/Downloads/SL-62-i383-2012-02-16-LiveCD.iso):
CODE
su -
dd if=/path-to-iso/SL-62-i383-2012-02-16-LiveCD.iso of=/dev/sdc


When the above command is finished you will have a bootable liveusb stick.


2. THE LOOP MOUNT, COPY AND INSTALL GRUB METHOD
This method allows for multiple partitions, allowing you to have 2 or more operating systems on one usb stick or use one partition for storage and one for the operating system. I am using a 4gb USB stick.

Insert the USB stick,
determine the device location using one of the above methods
Unmount any mounted partitions that are mounted from the USB stick.
Using the disk utility, format the usb stick
create a partition with the label “SL62” with the size equal to half the USB stick.
create a second partition with the label “Data” the size being the other half of the USB stick.
Mount the partition SL62 to the mount point /media/SL62 (you can just unplug and plugin the usb stick and it should mount both partitions for you in /media)
Create a mount point for the iso:
CODE
su -
mkdir /mnt/iso


Mount the livecd.iso and copy the data to the usb stick:
CODE
su -
mount -o loop /path-to-iso/ SL-62-i383-2012-02-16-LiveCD.iso /mnt/iso
cp -r /mnt/iso/* /media/SL62
unmount /mnt/iso


Install Grub on the USB stick:
CODE
su -
grub-install –root-directory=/media/SL62 /dev/sdc


Create a grub.conf file:
CODE
su -
gedit /media/SL62/boot/grub/grub.conf

and add the following stanza to the grub.conf file:
CODE
title SL62 i686
root (hd0,0)
kernel /isolinux/vmlinuz0 ro root=LABEL=SL62 rootfstype=auto liveimg
initrd /isolinux/initrd0.img


and click save and close. Unmount/eject the USB stick.

You now have a bootable usb stick with a data partition. Alternatively you could use the second partition for a second OS.



3. THE LIVECD-ISO-TO-DISK METHOD:
I use this method to put a install dvd to usb for computers without cd/dvd drives. With this method you can put livecd/dvds to usb as well.

Insert the USB stick,
determine the device location using one of the above methods
Unmount any mounted partitions that are mounted from the USB stick
CODE
su -
livecd-iso-to-disk –format –reset-mbr /path-to-iso/SL-62-i383-2012-02-16-LiveCD.iso /dev/sdc

Follow the onscreen instructions.

For more info on this method see: http://www.livecd.ethz.ch/usbdisk.html#livecd-iso-to-disk

4. THE LIVEUSB-CREATOR METHOD:
This method is also pretty easy and has built in functions to download a cd/dvd for you or use a iso that you have locally.

Insert the USB stick,
determine the device location using one of the above methods
CODE
su -
liveusb-creator –reset-mbr

Choose the target device, browse for a local iso or select one to download. Select persistent storage size if desired (this function does not always work)
Click the Create Live USB button and follow the instructions in the box.

For more info on this method see: http://www.livecd.ethz.ch/usbdisk.html https://fedorahosted.org/liveusb-creator/


Happy USB-ing cool.gif
©

Rufus Damaged my USB. Windows can not detect USB

I was able to fix the damaged USB by doing the following:

Go in Start > write “cmd” > right-click the Windows Command Prompt and choose to run it as an Administrator.
At the prompt, enter “DISKPART” to launch Microsoft’s disk management utility. It will take a second until it loads and when ready it will read “DISKPART>”.
Type in “list disk” to show a list of all disk drives. If your USB key is plugged into your PC, it should be listed here, along with other drives. Note the USB key’s disk number – you can pick it out by looking at the disk capacity.
Type “select disk n" (whatever is your USB disk number in place of "n").
Type “clean” for the utility to clean the disk, which DiskPart will confirm.
Create a new partition by entering “create partition primary”.
Choose this partition with “select partition 1", and then mark it as active by typing “active”.
Format the key by inputting “format fs=fat32". This should take a few minutes, and DiskPart will display a progress percentage.
Lastly, type “assign” to give this USB key a drive letter and “exit” to exit DiskPart.
©

January 29, 2013

Best alternative Linux desktops: 5 reviewed and rated

TechRadar

Best alternative Linux desktops: 5 reviewed and rated
Roundup See how good your Linux desktop could be
By Andrew Gregory from Linux Format Issue 166 January 27th
Comments

Best alternative Linux desktops: 5 reviewed and ratedIf you're not a fan of Ubuntu's switch to purple, try one of these alternative desktops instead
Related stories

How openness and collaboration helps Linux flourish
Google now considers itself a 'mobile first' company as YouTube soars
Linux in 2012: what to expect

The desktop on your Linux box used to stand for something very simple. If you were a KDE user, you valued control, power and the ability to customise.

In rough terms, if you used Gnome you wanted the desktop to get out of the way so you could get on with using your computer. If you used anything else, such as Xfce, LXDE or TekWM, you were running an ancient machine that would struggle with either of the big two of KDE and Gnome.

The change brought about by the release of KDE 4 changed all that. To compete, Gnome threw away its years of solidity for a new way of working; Unity arrived, with similar features to Gnome 3 but with the aim of tempting users away from Mac and Windows.

Brave as they were, these designs had much wrong with them, especially in the months following their release. Various products arrived to fill the gap left by the move away from the traditional desktops, some with the idea of refinement (Mint's Gnome Shell extensions and Cinnamon) and others aiming for a return to the old ways.

This desktop reformation and counter reformation has brought us to today's situation, and there has never been a better time to survey what's out there and what makes each project.

As they're easily the two biggest desktop projects, we've left out Gnome 3 and KDE 4, as well as Unity, but we've included Xfce to represent the more established desktops.
How we tested

We could have installed all of the desktops on the same Ubuntu box, testing them on the same files and applications, but that would have given Unity an unfair advantage. Likewise, Cinnamon and Mate would have had an unfair advantage had we picked a Mint box, as would Trinity and Razor-qt had we tested on Mandriva or another KDE-inflected Linux distribution.

So, we installed one distro per desktop. To get a feel for each desktop, we performed a range of tasks, including web browsing, copying files in a file browser and working on virtual desktops.

Desktops were rated according to how coherent the experience felt and how easy they were to work with. All our conclusions are subjective.
Desktops on test

Cinnamon
Mate
Razor-qt
Trinity
Xfce
Applications

Which desktop has the best suite of tools?

NemoThe Gnome team takes functions away from its file manager to make it easier to use on tablets; Cinnamon is busy putting them back in

The thing that distinguishes a desktop environment from a mere window manager is the suite of applications that it contains.

As the longest-established of the desktops we're looking at, you might expect Xfce to have the most complete set of tools, and it acquits itself well, with a native browser, Midori (available as part of the Xfce Goodies bundle), and a media player, Parole, which uses the GStreamer functionality. Xfce also includes a text editor, Mousepad, which is based on Leafpad.

Mate, as a fork of Gnome, also has the benefit of years of development. Nautilus (file manager), Gedit (text editor) and Eye of Gnome (image viewer) have been forked and renamed Caja, Pluma and Eye of Mate respectively, and offer identical (and even improved, in the case of Caja) functionality to their Gnome equivalents.

The Evince document viewer, Gnome Terminal and File Roller compression tool have also been forked to Mate, giving it a good, functional set of basic tools. While these are the applications that make up Mate officially, in reality every other Gnome app will work on it, so if you use Mate, there will always be an elegant solution to whatever you want to do.

Perhaps because of a desire to keep things as light as possible, the developers of Razor-qt don't bundle it with any software other than the bare desktop. They do, however, suggest a list of software that they see as being compatible with the project's aims, to complete a fully-featured Qt desktop.

These applications tend towards the fast and light, such as Clementine - a rewrite of Amarok before it was ported to KDE 4. There's also a strong preference for names that begin with the letter Q, as in QBitTorrent, QPDF view, QTwitter, QXMLEdit et al.

Trinity can be thought of as KDE 3 by another name. As such, it has a vast amount of native software, including the Konqueror web browser/ file manager. The aim of rolling the two functions in to one application was to blur the division between files on your machine, files on your network and files on the internet, and it worked fantastically well - so well that even the normally KDE-phobic members of Team LXF switch to KDE when they have to upload anything to our creaking servers. Quite why Konqueror was replaced with Dolphin in KDE 4 is a mystery to us, as even after nearly five years Konqueror is more useful, and its inclusion in Trinity is a massive boon.

As with Razor-qt, there's a reliance on one letter of the alphabet (in this case, K). Apart from looking silly, this makes things hard to find in an alphabetically-sorted menu; you find yourself having to read the name of each entry rather than just scanning down the first letter of the word.
Finding Nemo

As Cinnamon is a fork of the current version of Gnome, there's no need for the developers to fork their own versions of the applications; everything just works. But they have made their own modifications to the Nautilus file manager.

As the Gnome team repositions its desktop to a primarily tablet-oriented interface, it's reducing the number of features it has, to simplify the experience for people who will use the interface with swipes and finger-taps rather than mouse clicks.

Cinnamon has ditched Nautilus in favour of its own file manager, Nemo, which adds icons for searching for files, to toggle between file location and path bars to help you find your files, and buttons for Icon view, Compact view and List view. It's only really noticeable when you view the two file managers side by side (see image, left).
Verdict

Trinity: 5/5
Xfce: 4/5
Mate: 3/5
Cinnamon: 2/5
Razor-qt: 1/5

Trinity benefits from the huge number of features built in to KDE 3.
Appearance

These desktops put the graphical into Graphical User Interface

Good looks are as subjective in software as they are in any other sphere, but there are some things that we can all agree on. One is that Ubuntu has looked a heck of a lot nicer since it dropped the brown in favour of its current purple desktop background. The trouble is that many users also feel that it was easier to use in those days, regardless of how nice it looks.

When examining the appearance of the desktops on test, therefore, we're not just looking at fonts, colour, transparencies and animations; we want to glance at a desktop and know what things do without having to be told. We like desktop eye candy as much as the next Linux users, but by shunning KDE 4 and Gnome 3 we have implicitly signalled that it's not the end of the world if our machines don't look amazing.

What we do want to avoid, though, is outright ugliness, because there's no excuse for bad design.
Mate - 4/5

Mate

While we were testing the desktops, we installed Mate onto an Ubuntu 12.04 box, with frightful consequences. It was like stepping back into 2004; all sharp right angles, clunky icons and functional ugliness. That only made us appreciate how far things have come.

When installed as part of Linux Mint Debian edition, Mate takes on the theme of its host distro perfectly, reproducing all the shading and brushed metal effects that it shows when it's running Gnome. This isn't surprising; its window manager has simply been forked from Metacity and renamed Marco, so to all intents and purposes it's the same thing. Visually, the only thing that's changed is the grey gradient in Caja, the new file manager. Everything else - the system tray, the icons, the main menu button - is where it was when you left it two years ago. For an old Gnome user (or an old KDEer), there's no learning curve whatsoever.
Cinnamon - 5/5

Cinnamon

As you'd expect from a technology that was forked from Gnome 3, Cinnamon looks very similar. Shown here in its lesser-known Ubuntu habitat, it improves on the native Unity interface by having a system tray, which shows running apps and minimised windows, and keeping the toolbars at the top of the main application window, rather than using one global menu that changes depending on what app is in focus at the time.

Cinnamon also has a main menu in the bottom left of the screen, and while there is a search facility in there, your applications are still grouped in a menu structure, so you can choose for yourself whether to navigate the menus (useful if you can remember what something does but not what it's called) or type its name in the search box (useful only if you know what it's called).

It's also the shiniest of the desktops on test here, with some lovely transparency effects.
Trinity - 3/5

Trinity

For our money, Trinity isn't as pretty as the others in this Roundup, but that's not where its strengths lie. It's clearly laid out, a new user can switch it on and know what to do instantly, and it doesn't overload your eyes with pointless effects.

Of the five desktops, Trinity presents the user with the most obvious ways of changing the default appearance. As always when you're given loads of themes and config options, there are many more ways of getting it wrong than there are of getting it right, but the default is to have a main application menu in the bottom-left, a system tray and virtual desktops along the bottom, and it all makes perfect sense.

For migrating Windows 7 users scared by the lack of a Start button in Windows 8, this is the most sensible choice of desktop.
Xfce - 4/5

Xfce

Before Mate became the low-fat alternative desktop for Gnome users, Xfce was the low-fat alternative desktop for Gnome users. The graphical environment is built with GTK 2, which is the same toolkit used by Gnome up until the release of Gnome 3. This has the massive advantage that all those applications that were developed for Gnome work seamlessley in Xfce, with no need to load extra libraries to clog things up.

For the user, everything makes just as much sense as Trinity, with a main application menu, system tray, virtual desktops, applets and a notification area all along the top of the screen. The icons are smaller, so are either harder to identify or are less cluttered, according to taste.

One criticism of the interface is that some of the menus are cluttered with near-duplicate entries, which makes them hard to navigate.
Razor-qt - 3/5

Razor-qt

As Xfce is to Gnome, so Razor-qt is to KDE. That is, it uses the same libraries and follows the same behaviour that its heavier relative does, but it's a lot lighter and cleaner, and, by doing less, has less to confuse new users. You can see this KDE heritage in the default blue of the desktop, and in the widgets (Razor-qt uses the same graphical toolkit as KDE does; in this case, Qt).

Unfortunately, the look is a little rough around the edges - perhaps as a consequence of choosing the wrong window manager at install time (you're asked to make a decision between Xfwm and Metacity when you install Razor-qt, without being made aware of the significance of the choice - we chose Metacity). For a better-integrated look, it would make sense to go with KWin - so why give us the option of Metacity in the first place?
Documentation/support

The missing manuals are still out there, somewhere

Because they are intended as straightforward rewrites of old projects, Trinity and Mate are able to benefit from a huge amount of existing documentation. Even if you can't find the precise online HOWTO or FAQ that you need, the chances are that someone else will have asked exactly the same question in the many years of active development that Gnome 2 and KDE 3 had and will be able to point you in the right direction.

As always, Google is your friend. Trinity has its own documentation site, at bit.ly/XphGp9, for users and developers, while Mate's, at http://wiki.mate-desktop.org, is more focused at developers. It's pretty sparse, but if you need guidance with, say, Caja, you can search for the problem, substituting 'Caja' for 'Nautilus', and you'll find a solution.

Cinnamon doesn't have much for developers, but that's because Gnome 3 doesn't have much for developers. Users will have more luck, as the Linux Mint and Ubuntu forums are full of friendly people trying to help each other out. Making Cinnamon available on Ubuntu was a canny move, as it means the Ubuntu community has an interest in helping people to use it - and we call benefit.

The projects that are forks or continuations of previous desktops can rely on existing documentation. That can't be said about Razor-qt. It's the project that needs it the most, but the docs that we found were sparse at best.
Verdict

Xfce: 5/5
Trinity: 4/5
Mate: 3/5
Cinnamon: 3/5
Razor-qt: 1/5

The older projects have more reading matter, but they need it the least.
Add-ons

Bolt-on functionality to enhance the way you work

One way to expand the configuration options in Cinnamon is with dconf Editor (the Cinnamon equivalent to GConf Editor). This installs with an easy sudo apt-get install dconf-tools dconf Editor is a front-end to the configuration files that control how applications behave. This means that you're only one level removed from hacking the config files by hand, and that is why it isn't included by default.

Cinnamon also has an extensions and applets site on its project website, which is thriving. Xfce has a range of plugins for Thunar, which add more functions to the basic file manager.

While we like Konqueror's all-singing-all-dancing approach, we can also appreciate that not everybody wants to have to wade through features that they don't need, and would rather have a basic set that they can augment themselves. And speaking of KDE, Trinity has a long list of plugins on its site, but the site is in need of some design finesse - no matter, given that Trinity is already packed to the gunwhales with features.

Razor-qt takes a typically sane approach to plugins: right-click on an empty portion of the panel, choose what you want to add, then click Add Plugin. It's a bit fiddly to add a quick launcher, but the fact that they are so integrated with the desktop makes us think there will be a flood of third-party add-ons as soon as the desktop gains some more traction.
Verdict

Razor-qt: 3/5
Xfce: 2/5
Cinnamon: 2/5
Trinity: 1/5
Mate: 0/5

None of the candidates shine here, but it's not a high priority.
Configurability

Which desktops are easiest to mould to your image?

Trinity

Experienced KDE users won't be surprised to hear that the project that offers the most configuration options is the one most closely related to KDE. The config options start with Trinity before you've even finished the installation, in the shape of the excellent KPersonalizer config tool. This groups the hundreds of options in to a collection of sensible defaults. It gives you the choice of what language the interface will be in (we get a choice of US English and US English, which is entirely pointless - if you want more options, offer your services to the developers!).

You get to choose whether you want the GUI to behave like Mac OS, Windows, Unix or KDE, choose what level of eye candy you want according to the capabilities of your hardware, and pick a theme from a list of six options, including the default KDE style, the dated Keramik and the even more dated XP-alike Redmond theme. KPesonalizer concludes by informing you that all settings can be configured in the Trinity Control Centre.

In contrast, Mate, Cinnamon and Xfce do their best to provide sensible defaults that will work well for most people. There isn't such a heavy emphasis on making your own tweaks, but that's not to say that there's nothing you can do to adapt your working environment.

In Cinnamon, there's the System Tools > System Settings menu, which controls things such as themes, workspace behaviour and desktop effects, but these don't go as far as Trinity's config options.

Razor-qt is the other KDE-influenced desktop in this Roundup, and although it doesn't have much in the way of applications, the icon for its configuration tool is displayed prominently on the toolbar next to the main menu workspace switchers and Firefox button.

As with KDE, you get a fine degree of control over where different desktop areas go and how much space they take up: just right-click and choose Unlock Desktop to move widgets around, then choose Lock Desktop when you're done. As it is, there isn't much point in rearranging the desktop's appearance, but the ability to customise from the ground up bodes well for the future.

Options in Xfce and Mate are more limited, but that's not what they're about: sensible defaults are the order of the day.
Verdict

Trinity: 5/5
Raqzor-qt: 4/5
Xfce: 3/5
Cinnamon: 2/5
Mate: 2/5

Both Trinity and Razor-qt put tweaking at the heart of the user experience.
Installation

Features are no good if we can't get it running

Whatever your system, Xfce has been around for so long that there's bound to be a package available for your distribution. The other projects don't have the benefit of that longevity, so are understandably a little trickier to install.

Razor-qt, for instance, saw its first release in 2010, and thus far it has only made it into the default repository of Mageia. All other users will have to add a repository to their package manager, perform a system update and then install from the command line. It's a similar process with Cinnamon, which has been around only since the beginning of 2012.

Adding a repository and installing Mint from the command line is a straightforward task in Ubuntu Fedora and OpenSUSE, but OpenSUSE also gives you the option of a one-click installation (actually it's more like four clicks and a password, but it's still impressively smooth and quick). The last time we looked at Cinnamon, in a review of Cinnamon 1.2 in LXF157, the user was required to manually extract files into specific directories. Well done to the Mint team for your progress in this area!

Likewise, Mate requires that you add a repository, update and then install the matedesktop- environment metapackage, which brings in the latest versions of all the components.

Trinity, in contrast, caused us more problems. It has been almost five years since KDE 4 was released, which means it's almost five years since the first person tried it and wished that they could go back to KDE 3. Packages are available for Debian, Fedora, Ubuntu, Mandriva, Mageia and Slackware, putting it second only to Xfce in terms of distributions supported.

But once we'd gone through the installation procedure on Ubuntu 10.04 Long Term Support (we had to downgrade our installation, as Ubuntu 12.10 is not yet supported) and chosen a display manager, we were given a terse error message "kdeinit not working: check your installation" and dumped into a non-functioning desktop. In the end, we resorted to a live CD version, which is supremely easy, but inconvenient if you're not a fan of Fedora.
Verdict

Xfce: 5/5
Razor-qt: 4/5
Cinnamon: 4/5
Mate: 2/5
Trinity: 1/5

Only Xfce seems ready for mass-market adoption - the other projects have work to do.
The verdict

Cinnamon

What struck us most when testing these desktop environments is that the winner, if you'll excuse the cliche, is free software. We were expecting a massive gulf in usability between the newer offerings and the established likes of KDE 4, Gnome 3 and Unity.

We were ready to make allowances and make encouraging noises about it only being early days, but all five of the desktops are so good that this would only be patronising. Each criticism, therefore, should be taken as a compliment. The least accomplished DE here is unquestionably Razor-qt.

It has the fewest functions, but that's hardly a criticism, as the developers' primary aim is to keep it light and simple in a way that KDE 4 is not. In this aim, the team has succeeded massively. With more development, we'd love to see the traditional KDE distros offering this as an alternative option at login time, for KDE users with more modest hardware.

Mate and Xfce get the ultimate accolade in desktop usability: they just work. As Gnome 2 by another name, Mate has a stable codebase to draw on, loads of compatible applications and the polish to slot in seamlessly with your distro - as long as it's Mint.

Xfce has all this, but the fact that it is in the repositories of all the major and some of the minor distros means that it comes out ahead. We applaud the Mate team for their efforts, but for a lightweight, full-featured Gnome alternative we'd still plump for Xfce.
Everyone's a winner

The surprise package has to be Trinity. It's a note-perfect reimplementation of KDE 3, which means that it's full of features, it looks good, it has tons of software and it will run passably well on old hardware.

If you don't think you like KDE, try Trinity and there's a good chance that it will change your mind. Which leaves us with Cinnamon. When we first tried it in LXF157 it felt half-baked, but the last few releases have knocked the rough edges off and provided a seamless fusion of Gnome 3 glitz and Gnome 2 usability. Linux Mint wins again.
1st: Cinnamon

Web:http://cinnamon.linuxmint.com/
Helps you get to the applications without getting in your way.
2nd: Trinity

Web:www.trinitydesktop.org
KDE sceptics will be amazed, not least by the usable default settings.
3rd: Xfce

Web:www.xfce.org
Full-featured, stable and the best choice for 3D unbelievers.
4th: Mate

Web:http://mate-desktop.org
Occupies a niche that has already been filled, but usable nonetheless.
5th: Razor-qt

Web:http://razor-qt.org
Bags of potential as a KDE alternative, but not ready for mainstream use.
©