Project Idea: X@home

 
I would like to create a platform (x@home) that allows users to volunteer their free computer time to solving computationally intensive problems similar to the folding@home and seti@home projects. With the two mentioned @home projects, you subscribe to each problem individually, but this project is based around the idea of creating a main server that enables the user to subscribe to various different problems with ease.

The goal of this project is to create an infrastructure that can benefit problems of any sort that can use distributed computing. Thus, the problems this project works on will vary depending on the user-bases interests. The problems ideal for this site will need a few days' computation time, as these are substantial projects that will not monopolize the system and will allow for a volunteered computer to help out with multiple projects.

Volunteered computers will download a program that runs in the background, ascertains when the user's computer is idle and then pings the main server to get details about current problems and starts to work on one of the user's chosen problems. This will primarily involve process parallelism, but thread parallelism will be tested to see if it can be efficient over the internet.

Any Unix-based machine with ssh and openMPI should be able to subscribe to the cluster. I will be benchmarking the project with particles code (generates particles in a disk and performs computationally intensive operations on them: mean, std deviation, etc.) and a prime number generator.

Notes on setting up the cluster

[jcotham@radium ~]$ cat /proc/cupuinfo

jcotham@radium:~[jcotham@radium ~]$ tar xvf hello.tar

bash: make: command not found
jcotham@radium:~[jcotham@radium ~]$ which make

jcotham@radium:~[jcotham@radium ~]$ man script
root@radium:~[root@radium ~]# sudo yum install make.x86_64
root@radium:~[root@radium ~]# yum install openmpi-devel

jcotham@radium:~[jcotham@radium ~]$ rpm -qa --filesbypkg |grep mpicxx

export PATH=${PATH}:/usr/lib64/openmpi/bin/

[root@radium ~]# wget https://newton.utk.edu/workshop/hello.tar

root@radium:~/hello[root@radium hello]# make
mpicxx -i-dynamic -c hello.cc
--------------------------------------------------------------------------
The Open MPI wrapper compiler was unable to find the specified compiler
g++ in your PATH.

Note that this compiler was either specified at configure time or in
one of several possible environment variables.
--------------------------------------------------------------------------
make: *** [hello.o] Error 1

yum provides */g++

root@radium:~[root@radium ~]# yum install gcc-c++
root@radium:~/hello[root@radium hello]# make
mpicxx -i-dynamic -c hello.cc
cc1plus: error: unrecognized command line option "-i-dynamic"
make: *** [hello.o] Error 1
root@radium:~/hello[root@radium hello]# vi Makefile
root@radium:~/hello[root@radium hello]# make
mpicxx  -c hello.cc
mpicxx  -o hello hello.o
root@radium:~/hello[root@radium hello]# ./hello
./hello: error while loading shared libraries: libmpi_cxx.so.1: cannot open shared object file: No such file or directory
root@radium:~/hello[root@radium hello]# which mpicxx
/usr/lib64/openmpi/bin/mpicxx
[root@radium hello]# cd /usr/lib64/openmpi/
[mroot@radium:/usr/lib64/openmpi[root@radium openmpi]# cd lib
[mroot@radium:/usr/lib64/openmpi/lib[root@radium lib]# pushd ~
~ /usr/lib64/openmpi/lib
root@radium:~[root@radium ~]# vi .bashrc

export PATH=${PATH}:/usr/lib64/openmpi/bin/

root@radium:~[root@radium ~]# echo $LD_LIBRARY [K_PATH
:/usr/lib64/openmpi/lib/
/usr/lib64/openmpi/lib

root@radium:~/hello[root@radium hello]# ./hello
Hello World! I am 0 of 1
root@radium:~/hello[root@radium hello]# mpirun -n 2 hello
Hello World! I am 0 of 2
Hello World! I am 1 of 2

Survey

Name (Lastname | Firstname)

Cotham

John

1. Prior to this semester, how many parallel programs had you written?

I have written a few programs with threads, but nothing that I've specified to run on different cores or computers.

2. Prior to this semester, how many CPUs (or cores) had you run your code on?

I have written a few programs with threads, but nothing that I've specified to run on different cores or computers.

3. What kind of work do you plan to do after you graduate?

I'm interested in data visualization, but I'm not sure what's about to happen.

4. What, in your view, is the future importance of parallel programming?

It seems to be the way forward. New materials like graphene may allow us to power up the Ghz a little bit, but the largest gains in speed are likely to come from more and more transisters and more parallelization.

5. What, in your view, will be the importance of parallel programming in your future work?

I doubt I'll be parallelizing code very much by hand, but who knows? I think it will be a good skill to have in the back pocket.

6. What makes UT a unique/special place for students to learn parallel programming?

There seems to be plenty of resources to explore parallel programming with Newton and the new clusters we set up.

Assignment 2 - Particles

http://web.eecs.utk.edu/~jcotham/cs462/parallelProg02/parallelProg02.tgz
http://cosc462.newton.utk.edu/jcotham/parallelProg02.tgz

Sample outputs:

numParticles 10000
shape disk
radius 1.000000
xMean 0.007981
yMean 0.001801
xVariance 0.251990
yVariance 0.249922
distanceMean 0.907270
distanceVariance 0.501912
distanceInverseSquaredMean 19.511139
distanceInverseSquaredVariance 22907.612434


numParticles 10000
shape square
radius 1.000000
xMean 0.002533
yMean 0.006514
xVariance 0.330165
yVariance 0.334416
distanceMean 1.041235
distanceVariance 0.664582
distanceInverseSquaredMean 15.483685
distanceInverseSquaredVariance 28928.666000


For both the square and the disk, the xMean and yMean should be almost 0, as the shapes are centered at the origin. Thus, the sample values are about what is expected. The xVariance and yVariance are both consistent for each shape, as expected, and the variance is smaller for the circle than for the square, which makes sense.

The lecture notes cite the average pairwise distance for particles in a circle to be about .90541*r. This is very close to my mean of .907. The distanceMean for the square is close but slightly larger, as would be expected. The variance seems reasonable.

I'm not really sure what the distanceInverseSquaredMean is supposed to measure, but the values are close to the sample values. The distanceInverseSquaredVariance seems to be way off, but I'm not sure what might be going on with it.