ssh

• avoid automatic time outs on sessions
• GSSAPIAuthentication no
• GSSAPIDelegateCredentials no
• ServerAliveInterval 60

• to run something on different machines:
• for i in `seq 17 19`; do ssh xml$i sed -i '"s/boost_clicks=0.0/boost_clicks=10/"' my-search/depot/search/Whatever/etc/*/rank.conf ; done
• 
  
for i in 2 3 4 7 8 9 10 11 12 13 14 15 16
do
echo xml$i
ssh toto@xml$i cat /etc/debian_version
done

• ssh users are Ubuntu users. There are no specific ssh users

• to do ssh without entering a pw

• ssh-copy-id user@server

Python data types

------------------------ built in types

-- tuples (immutables)
> empty = ()
> t = 12345, 54321, 'hello!'
or
> t = (12345, 54321, 'hello!')
> t
(12345, 54321, 'hello!')

-- set

> a=set()

> a.add("orange")

> a

{'orange'}

> basket = {'apple', 'orange', 'apple', 'pear', 'orange', 'banana'}      # creates a set without duplicates
> basket.add("'orange")
> basket
{'orange', 'pear', 'apple', 'banana'}

>

> my_data = {0,0}

> my_data

{0}

-- list  (might contain different types, duplicates. It is mutable)

z_list = []
my_list = ["apple", "banana", "cherry"]
my_list.append("pear")
a_list=[0]
a_list = [66.25, 333, 333, 1, 1234.5]
a_list = ['foo', 'bar', 'bar']
a_list = [[ 4.9,  3. ,  1.4,  0.2], [ 4.9,  3. ,  1.4,  0.2]]

-- dictionaries (have keys)

> my_dict = {}

> d={0:0}

> d

{0: 0}

tel = {'jack': 4098, 'sape': 4139}
tel['john'] = 4444.  # add
print (tel.keys())         # returns a list

  dict_keys(['jack', 'sape', 'john'])

print (tel.values())      # returns a list

  dict_values([4098, 4139, 4444])

https://docs.python.org/3.6/library/datatypes.html

Use
>>> globals()

to see all variables.

Joel Test score

The Joel Test is a twelve-question measure of the quality of a software team.

• Do you use source control?
• Can you make a build in one step?
• Do you make daily builds?
• Do you have a bug database?
• Do you fix bugs before writing new code?
• Do you have an up-to-date schedule?
• Do you have a spec?
• Do programmers have quiet working conditions?
• Do you use the best tools money can buy?
• Do you have testers?
• Do new candidates write code during their interview?
• Do you do hallway usability testing?

regex

^ Matches the starting position within the string
$ Matches the ending position within the string


^10[1-9]{6}$    ("10" followed by 6 digits)
^10.{6}$           ("10" followed by any 6 characters)

• mystring/   followed by two or three digits
• sed -r 's/.*mystring\/([0-9]{2,3}).*/\1/g'

.

apache nutch

apache-nutch-1.11
--------------------------- one shot
./bin/nutch parsechecker -dumpText http://www.deco.com > out.txt
./bin/nutch parsechecker -dumpText http://www.deco.com/cuisine/vos-visites |grep tc

wget -O - 'http://www.deco.com/cuisine/vos-visites' |grep tc_vars.http

--------------------------- crawl
./bin/crawl urls/seeds.txt out-crawl 5
(look at what the crawl script does, it is basically a loop calling bin/nutch)

./bin/nutch fetch 5-out-crawl/segments/20160717060128/ -noParsing  (the one did in crawl)

--------------------------- read /crawl db
./bin/nutch readdb out-crawl/crawldb -stats
./bin/nutch readdb 4-out-crawl/crawldb -dump 4-out-crawl-db-dump-20160512 -format csv -topN 1000

--------------------------- read /segments
./bin/nutch readseg -dump 4-out-crawl/segments/20160716234706 4-out-crawl-readseg-20160716234706
grep -ir outlink 3-out-crawl-readseg  --color  |wc

--------------------------- read one key from  a segment
./bin/nutch readseg -get 4-out-crawl/segments/20160717025923  "http://www.moulinex.fr/Cuisson/Croques-%26-Gaufres/c/waffle%2Bmakers"  -nocontent -noparse -noparsetext

--------------------------- read HTML from one key from  a segment
./bin/nutch readseg -get 2-out-crawl-wiki-nikon-num-round-2/segments/20160823085007/  "https://en.wikipedia.org/wiki/Nikon" -nofetch -nogenerate -noparse -noparsedata -noparsetext > Nikon.html

--------------------------- doc
https://wiki.apache.org/nutch/CommandLineOptions

--------------------------- plugin
bin/nutch plugin   parse-html   org.apache.nutch.parse.html.HtmlParser   index2.htm
https://wiki.apache.org/nutch/bin/nutch%20plugin

.
.

docker survival kit

on Ubuntu 14.04:

-------------- docker service itself

service docker start

-------------- install

sudo bash
vi /etc/apt/sources.list.d/docker.list
deb https://apt.dockerproject.org/repo ubuntu-trusty main

apt-get update
you may have to do this with the associated keys if the former doesn't work
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys F76221572C52609D

apt-get -y install docker.io
apt-get install docker-engine
(don't do    "apt-get install docker")

ln -sf /usr/bin/docker /usr/local/bin/docker

sed -i '$acomplete -F _docker docker' /etc/bash_completion.d/docker.io
or
sed -i '$acomplete -F _docker docker' /etc/bash_completion.d/docker
(depends which works)

• start docker daemon
• $ sudo service docker start

docker -v    (should be at least 1.12)
docker version
docker info
docker pull hello-world
docker run -it  hello-world

docker pull centos:6.6
docker run -it centos:6.6 /bin/bash

sudo docker images -a

------------- Main

• Run but exits right away with an error. Still, creates a container.
• docker run ubuntu
• Run and keeps an interactive session
• docker run -it ubuntu /bin/bash
• docker run -it alpine /bin/sh
• sudo docker run -it {repository_name}:{tag} /bin/bash
• Execute in a running container:
• “docker exec -it 7b33f4ed03a7 /bin/bash” (or /bin/sh on alpine)
• sudo docker exec -it $(sudo docker ps -l -q) env
• with root privilege in the docker bash session
• sudo docker exec -it --user root 58d3daf3a0dd /bin/bash
• Restart an existing container to continue working in it
• docker start 912ccdad8380
• then docker exec -it 912ccdad8380 /bin/bash

• To build a container from a Dockerfile:
• sudo docker build {Dockerfile_path}

------------- 

docker rename 7d8b244480a2 my_django____4         (rename a container)
docker commit 7d8b244480a2  bob/hello_django:version2  (commit a new image from a running container)

docker save -o /home/bob/bob-hello-django-version4.tar  bob/hello_django:version2

docker load -i bob-hello-django-version2.tar

-------------

- "ls /etc/*release"  to know the container OS.

• remove all containers and clear cache (do not remove images...)
• docker rm $(docker ps -a -q) && docker ps -a | cut -c-12 | xargs docker rm

• remove a specific image 
• sudo docker rmi -f 4f8eceb5cb78
• WARNING WATCH OUT: remove all images
• docker rmi $(docker images -q) -f
• ignore container entrypoint:
• docker run -it --entrypoint=/bin/bash michal/hello_django_srp:version5

------------- problem 1

Docker version 1.6.2, build 7c8fca2

> docker save -o /tmp/docker1.tar  michal/hello_django_srp:version6

FATA[0000] Error response from daemon: could not find image: no such id:  michal/hello_django_srp:version6 

--> there are two spaces before "michal" and docker is not robust to this

------------- static IP address

docker network create --subnet=172.18.0.0/16 mynet18 docker run --net mynet18 --ip 172.18.0.22 -it ubuntu:14.04 bash

------------- bind volume mount (you need data from the host machine)

docker run -it -v /home/lee/tmp/docker-bind:/opt/container-path bob/solr_6_6:v7

otherwise, you can simply copy from the host to the container:

docker cp ./syn_acc.txt dfccf5a493c3:/usr/share/elasticsearch/config/resources/v1_02/

------------- other commands

docker ps --last 5

docker inspect 7832d56ddf56

docker logs 7fbc2e820d4c 

docker run -it -p 8580:8080 bob/my-jetty:v1 (port matching)

-------------- when using rancher Desktop

 You can check index creation by doing
 "watch docker ps"   (get docker port number)

  CONTAINER ID   IMAGE                                                                    COMMAND                          PORTS                                                                                                                                NAMES
 d229f1ce7bc2   localhost/testcontainers/m3ancimbepv8mj4a:latest   "/bin/tini -- /usr/l…"     0.0.0.0:49196->9200/tcp, :::49196->9200/tcp, 0.0.0.0:49195->9300/tcp, :::49195->9300             tcp   sleepy_galois
 5d89fb8008f2   testcontainers/ryuk:0.11.0                                            "/bin/ryuk"                  0.0.0.0:49194->8080/tcp, :::49194->8080/tcp                                                                                testcontainers-ryuk-5cede40f-ba07-41f2-a324-6f155ae34fa8

.
.

strange colors on vlc / Windows 7 / need nvidia driver

http://fr.download.nvidia.com/Windows/341.92/341.92-notebook-win8-win7-64bit-international.exe