Some Python packages

• http://www.scipy.org/
• ecosystem regrouping   numpy, matplotlib, pandas
• http://www.numpy.org/
• array processing for numbers, strings, records
• http://matplotlib.org/
• ploting package
• http://pandas.pydata.org/
• data analysis
• anaconda
• https://www.continuum.io/downloads
• 387 packages
• numpy, matplotlib, pandas, scipy

yum install python27

---- install numpy
yum install python27-pip python27-devel parallel
cd /usr/bin
ln -s pip2.7 pip
pip install --upgrade pip
pip install requests solrpy
yum install gcc
pip install numpy

---- numpy, scipy on ubuntu
yum install python-pip python-dev build-essential
yum install libatlas-base-dev gfortran
pip install numpy scipy

---- scipy on centOS
yum install lapack-devel
pip install scipy

---- Install pip

yum install python-pip
if you have an error like: "urllib3/util/ssl_.py:90: InsecurePlatformWarning" do:

sudo yum remove python-openssl

---- look for a package:
pip search requests |grep requests
---- see if it is installed
pip show requests
---- see all installed
pip freeze

---- MySQL
pip install MySQL-python       didn't work right away. I had to do this

yum search all MySQL
yum install MySQL-client
yum install MySQL-devel
yum install MySQL-python27

---- image module, PIL
pip install PIL --allow-external PIL --allow-unverified PIL

---- image hash
pip install ImageHash

--- look for local Python package
pip list |grep cv

--- Solr sunburnt (Python Solr interface)

You may have errors with the default lib. What you need to do:
sudo pip uninstall sunburnt
git clone http://github.com/tow/sunburnt.git

cd sunburnt

sudo python setup.py install

--- SolrPy
usually better than sunburnt

--- virtualenv gives a urllib3 error

virtualenv my-virtualenv -p /usr/bin/python

my-virtualenv$ source bin/activate

(my-virtualenv) lee@mycomp:~/python/flask/my-virtualenv$ deactivate

.

.

Postfix survival kit (SMTP server)

http://www.postfix.org/

config in:
/etc/postfix/main.cf

Traffic acquisition

3 main players:

• http://www.criteo.com/
• http://www.taboola.com/
• http://www.outbrain.com/fr

curl survival kit

curl --request PUT --user admin http://maven.toot.com/nexus/content/repositories/releases/com/leguide/search3s/search-3S/3.5.4/threesidx4/lg_platform_scheduler_server.conf.xml --data-binary @/home/lee/lee_dev/search-platform/platform-scripts/config-prod/price_intelligence/threesidx4-cluster/lg_platform_scheduler_server.conf.xml

apt-get survival kit

apt-cache policy gcc-4.8-base
apt-get --simulate install libgfortran3
apt-get -f install       (fix broken)
apt-get update
ppa purge

aptitude install libgfortran3    : automatically downgrade packages

openvpn

  ps aux |grep vpn
  sudo /etc/init.d/openvpn start
  sudo /etc/init.d/openvpn stop
  /sbin/ifconfig

Using Latex online

.
.
http://www.sciweavers.org/free-online-latex-equation-editor

You can cut and paste things like

\flushleft

  x(u) = \begin{bmatrix}1 \\ 1 \\ 0 \end{bmatrix} \\

  y(i) = \begin{bmatrix}0 \\ 1 \\ 0 \end{bmatrix} \\

\centering\\

\flushleft

preference: p(u,i) = x^\intercal(u) * y(i)\\

\centering\\

\flushleft

min  \sum\limits_{u,i} (r(u,i) - p(u,i))^2 +  \lambda (||x(u)||^2 + ||y(i)||^2)

nomachine

• logs on the server

/usr/NX/var/log

• kill the sessions

sudo nxserver --terminate   myusername

• restart nxserver
• sudo service nxserver stop
• sudo service nxserver start

X server and nvidia

• To stop the X session
• Hit CTRL+ALT+F1 and login using your credentials.
• kill your current X server session by typing 
• sudo service lightdm stop or sudo stop lightdm
•  If you want to reinstall the nvidia exe
• Enter runlevel 3 by typing 
• sudo init 3 and install your *.run file.
•  You might be required to reboot when the installation finishes. If not, run
• "sudo service lightdm start"  or "sudo start lightdm" to start your X server again.
• If you haven't stop the X server at all, you can come back to the X server environment from the command line environment by doing:
• Hit CTRL+ALT+F7 and login using your credentials.

puppet

• to check if the puppet agent is running:
•   service puppet status
• usually, the service is monitoreed and should not be stopped
• Enable / disable
• [root@p1al25]# puppet agent --disable "Modification blahblah" --verbose
• [root@p1al25]# puppet agent --enable --verbose
• Execute puppet agent
• puppet agent -t  --debug
• used to install the latest version of the package
• check if the agent is enabled or disabled
• [root@p1al25 ~]# service puppet status
• puppet (pid 4387) is running...
• [root@p1al25 ~]# puppet agent --disable "my message" --verbose
• check if it is disabled or not
• [root@p1al25 ~]# cat `puppet agent --configprint agent_disabled_lockfile` 
• {"disabled_message":"reason not specified"}
• I advise to use --verbose, otherwise you don’t get the errors.
• In the case herebelow, I was using puppet with a non root user. This user doesn’t have the privilege to create files in its root.
• bash-4.1$ puppet agent
• bash-4.1$ puppet agent --verbose
• Error: Could not set 'directory' on ensure: Permission denied - /data1/toto/.puppet
• Error: Could not set 'directory' on ensure: Permission denied - /data1/toto/.puppet
• Wrapped exception:
• Permission denied - /data1/leguide/ciao/.puppet
• puppet conf
• puppet config print
• is supposed to print what is in
• less /etc/puppet/puppet.conf
• puppet certificates
• server:
• puppet cert clean lgstagevirt-vm-3s-1.toto.com
• puppet cert list --all |grep vm
• client:
• \rm -r /var/lib/puppet/ssl
• puppet agent -t -d --waitforcert 30
• then server:
• puppet cert sign  lgstagevirt-vm-3s-1.toto.com