A New Home

The Argo Database web application is getting a new home at argoweb.whoi.edu! This post documents the changes made to Argoweb.

Changes to Argoweb

Installation of MySQL Database and Apache Tomcat was accomplished using a custom bash shell script to download, unzip, compile the applications from source. The following changes were made to argoweb:

• added groups 'argoweb', 'mysql', and 'tomcat'
• added users 'argoweb', 'mysql', and 'tomcat'
• removed:
• apache2 (replaced by Tomcat)
• installed:
• vim 
• openjdk 6
• g++ 4.4
• MySQL 5.1
• Apache Tomcat 6 Apache Tomcat 7 
• Note: Tomcat 6 does not support the latest version of Java. Tomcat 7, however, is not natively supported by Debian so it must be installed from source.
• used iptables to route incoming traffic on port 80 to the Tomcat server:

root@argoweb:/etc/tomcat6# iptables -t nat -I PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080
root@argoweb:/etc/tomcat6# iptables-save
# Generated by iptables-save v1.4.8 on Thu May 3 05:31:30 2012
*nat
:PREROUTING ACCEPT [53:7910]
:POSTROUTING ACCEPT [2:191]
:OUTPUT ACCEPT [2:191]
-A PREROUTING -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8080
COMMIT
# Completed on Thu May 3 05:31:30 2012

• performed dump of previous database :

mysqldump -u root -p --databases floats > argoweb_dump.sql

• used sql file to populate Argoweb database
• deployed the Argo Database application on the Tomcat server
• created a script to mirror some directories from Ursa to ArgoWeb (/usr/local/bin/ursa-volume-sync):
• ursa:/Volumes/U1/argo -> argoweb:/data/Volumes/U1/argo
• ursa:/Volumes/U2 -> argoweb:/data/Volumes/U2
• cron'd to run twice daily. After the initial transfer, this script has completed in under 10 minutes, without any major changes to files on Ursa. After an update to the GDAC, it may take significantly longer.

The following script was used to install MySQL database and Apache Tomcat from source:

1. #!/bin/bash
3. ##########################################
4. # MYSQL AND APACHE TOMCAT INSTALL SCRIPT #
5. ##########################################
6. #
7. # AUTHOR
8. #   Alex Ekholm, aekholm@whoi.edu, Woods Hole Oceanographic Institution
9. #
10. # CREATED
11. #   Fri Apr 27, 2012
12. #
13. # NOTES
14. #   - This script can be used to automate installation of the following applications from source:
15. #       1) MySQL 5.1.62
16. #       2) Apache Tomcat 7.0.27
17. #   - Will attempt to download source code automatically if not present.
18. #
19. # REQUIREMENTS
20. #   - Must be run as root user
21. #   - wget, getopts, gcc, g++, make, ant
22. #
23. #
25. ##########################################
26. # CONFIGURATION              #
27. ##########################################
28. #
29. # Relevent files, directories, and other misc. install options are specified here.
30. #
32. # MySQL configuration
33. mysql_home="/home/mysql"
34. mysql_prefix="/usr/share/mysql"
35. mysql_dir="$mysql_prefix/mysql-5.1.62"
36. mysql_tar="$mysql_dir.tar.gz"
37. mysql_config_opts="--prefix=$mysql_prefix"
39. # Apache Tomcat configuration
40. tomcat_home="/home/tomcat"
41. tomcat_dir="./tomcat/apache-tomcat-7.0.27-src"
42. tomcat_tar="$tomcat_dir.tar.gz"
43. tomcat_prefix="/usr/share/tomcat7"
44. tomcat_config_opts="base.path=$tomcat_prefix"
46. # tar options
47. TAROPTS="-xzf"
49. # debug - set DEBUG=1 to display debug info
50. _DEBUG=0
52. # verbose operation
53. VERBOSE=$_DEBUG
55. # working dir - default is current dir
56. src_dir="/usr/local/src"
58. # mirrors
59. mysql_mirrors=(
60.     "http://mysql.he.net/Downloads/MySQL-5.1/mysql-5.1.62.tar.gz"
61.     "http://mysql.mirrors.hoobly.com/Downloads/MySQL-5.1/mysql-5.1.62.tar.gz"
62.     "http://mirror.services.wisc.edu/mysql/Downloads/MySQL-5.1/mysql-5.1.62.tar.gz")
63. tomcat_mirrors=(
64.     "http://www.fightrice.com/mirrors/apache/tomcat/tomcat-7/v7.0.27/src/apache-tomcat-7.0.27-src.tar.gz"
65.     "http://mirrors.ibiblio.org/apache/tomcat/tomcat-7/v7.0.27/src/apache-tomcat-7.0.27-src.tar.gz"
66.     "http://mirror.candidhosting.com/pub/apache/tomcat/tomcat-7/v7.0.27/src/apache-tomcat-7.0.27-src.tar.gz")
68. ##########################################
69. # INSTALL MYSQL DATABASE         #
70. ##########################################
71. function install_mysql {
73.     # check directory
74.     if [ ! -d "mysql" ]; then
75.         mkdir mysql
76.     fi
77.    
78.     # download source if necessary
79.     ask "Download MySQL source?" Y && if [ ! -f "$mysql_tar" ]; then
80.         # try each mirror until success
81.         for mirror in "${mysql_mirrors[@]}"; do
82.             # download source
83.             WGETOPTS="-P mysql $mirror"
84.             dbg "executing wget: 'wget $WGETOPTS'"
85.             wget $VOPT $WGETOPTS
86.            
87.             if [ -f "$mysql_tar" ]; then break; fi
88.         done
89.        
90.         # check
91.         if [ ! -f "$mysql_tar" ]; then
92.             echo "Failed to download MySQL source: $mysql_tar"
93.             exit 1
94.         fi
95.     fi
97.     # create group
98.     ask "Create group 'mysql'?" Y && groupadd mysql
100.     # create user
101.     ask "Create user 'mysql'?" Y && useradd -b $mysql_home \
102.         -d $mysql_home \
103.         -g mysql \
104.         mysql
106.     # unzip
107.     ask "Unzip MySQL database?" Y && tar $TAROPTS $mysql_tar -C mysql
108.    
109.     # change to mysql dir
110.     chdir $mysql_dir
112.     # configure
113.     verbose "MySQL Configuration options: '$mysql_config_opts'"
114.     ask "Configure MySQL?" Y && ./configure $mysql_config_opts
115.        
116.     # compile
117.     ask "Make clean?" Y && make clean
118.     ask "Compile MySQL?" Y && make
119.     ask "Compile MySQL installation modules?" Y && make install
121.     # postinstallation setup
122.     chdir $mysql_prefix
123.     chown -R mysql .
124.     chgrp -R mysql .
125.     ask "Install MySQL?" Y && ./bin/mysql_install_db --user=mysql --basedir=$mysql_prefix
126.     chown -R root .
127.     chown -R mysql var
129.     # optional 
130.     ask "Copy support files?" N && cp support-files/my-medium.cnf /etc/my.cnf
132.     # start mysql daemon
133.     if ask "Start MySQL daemon?" Y ; then
134.         ./bin/mysqld_safe --user=mysql --basedir=$mysql_prefix &
135.     fi
137.     # return to root dir
138.     chdir $src_dir
140. } # install_mysql
142. ##########################################
143. # INSTALL APACHE TOMCAT          #
144. ##########################################
145. function install_tomcat {
146.     # check directory
147.     if [ ! -d "tomcat" ]; then
148.         mkdir tomcat
149.     fi
150.    
151.     # download source if necessary
152.     ask "Download Apache Tomcat source?" Y && if [ ! -f "$tomcat_tar" ]; then
153.         # try each mirror until success
154.         for mirror in "${tomcat_mirrors[@]}"; do
155.             # download source
156.             WGETOPTS="$VOPT -P tomcat $mirror"
157.             dbg "wget $WGETOPTS"
158.             wget $WGETOPTS
159.            
160.             if [ -f "$tomcat_tar" ]; then break; fi
161.         done
162.        
163.         # check
164.         if [ ! -f "$tomcat_tar" ]; then
165.             echo "Failed to download Apache Tomcat source: $tomcat_tar"
166.             exit 1
167.         fi
168.     fi
170.     # create group
171.     ask "Create group 'tomcat'?" Y && groupadd tomcat
173.     # create user
174.     ask "Create user 'tomcat'?" Y &&  useradd -b $tomcat_home \
175.         -d $tomcat_home \
176.         -g tomcat \
177.         tomcat
180.     # unzip
181.     ask "Unzip Apache Tomcat Server?" Y && tar $TAROPTS $tomcat_tar -C tomcat
183.     # change to tomcat dir
184.     chdir $tomcat_dir
186.     # configure tomcat
187.     # cd echo "$tomcat_config_opts" > build.properties
188.    
189.     # compile tomcat
190.     ask "Compile Apache Tomcat Server from source?" Y && ant
192.     # create directories
193.     if ask "Create directory structure?" Y ; then
194.         # /var/lib
195.         if [ ! -d /var/lib/tomcat7 ] ; then
196.             mkdir -p /var/lib/tomcat7/common/classes \
197.                  /var/lib/tomcat7/server/classes \
198.                  /var/lib/tomcat7/shared/classes
199.         fi
200.        
201.         # /var/log
202.         if [ ! -d /var/log/tomcat7 ] ; then
203.             mkdir /var/log/tomcat7
204.         fi
205.        
206.         # /var/cache
207.         if [ ! -d /var/cache/tomcat7 ] ; then
208.             mkdir /var/cache/tomcat7
209.         fi
211.         # /etc - conf
212.         if [ ! -d /etc/tomcat7 ] ; then
213.             mkdir /etc/tomcat7
214.         fi
215.     fi
216.    
217.     # copy compiled files
218.     if ask "Copy compiled files to install directory?" Y ; then
219.         # /usr/share/
220.         cp -r output/build/bin $tomcat_prefix/
221.         cp -r output/build/lib $tomcat_prefix/
222.        
223.         # /etc
224.         cp -r output/build/conf/* /etc/tomcat7
225.        
226.         # /var/lib
227.         cp -r output/build/webapps /var/lib/tomcat7/webapps
228.         ln -s /etc/tomcat7 /var/lib/tomcat7/conf
229.         ln -s /var/log/tomcat7 /var/lib/tomcat7/logs
230.         ln -s /var/cache/tomcat7 /var/lib/tomcat7/work
231.        
232.         # permissions
233.         chown tomcat /var/log/tomcat7 -R
234.         chgrp tomcat /var/log/tomcat7 -R
235.         chown tomcat /var/lib/tomcat7 -R
236.         chgrp tomcat /var/lib/tomcat7 -R
237.         chown tomcat /var/cache/tomcat7 -R
238.         chgrp tomcat /var/cache/tomcat7 -R
239.         chown tomcat /etc/tomcat7 -R
240.         chgrp tomcat /etc/tomcat7 -R
241.         chown tomcat /usr/share/tomcat7 -R
242.         chgrp tomcat /usr/share/tomcat7 -R
243.     fi
245.     # return
246.     chdir $src_dir
248. } # install_tomcat
250. ##########################################
251. # PROMPT FUNCTION            #
252. ##########################################
253. #
254. # USAGE
255. #   (long version)
256. #       if [ ask "<prompt>" <optional-default> ]; then
257. #           # Control will enter here if yes
258. #       fi
259. #
260. #   (short version)
261. #       ask "<prompt>" && <command-to-exec-if-yes>
262. #       ask "<prompt>" <default> && <command-to-exec-if-yes>
263. #       ask "<prompt>" <default> || <command-to-exec-if-no>
264. #    
265. # EXAMPLE
266. #   ask "Install MySQL database?" Y && install_mysql
267. #
268. # RETURN VALUE
269. #   0 if user responds YES
270. #   1 if user responds NO
271. #
272. function ask {
273.     while true; do
275.     # Assign prompt and default
276.         if [ "${2:-}" = "Y" ]; then
277.             prompt="Y/n"
278.             default=Y
279.         elif [ "${2:-}" = "N" ]; then
280.             prompt="y/N"
281.             default=N
282.         else
283.             prompt="y/n"
284.             default=
285.         fi
287.         # Ask the question
288.         read -p "$1 [$prompt] " REPLY
290.         # Default
291.         if [ -z "$REPLY" ]; then
292.             REPLY=$default
293.         fi
295.         # Return only if the reply is valid
296.         case "$REPLY" in
297.             Y*|y*) return 0 ;;
298.             N*|n*) return 1 ;;
299.         esac
301.     done
302. } # ask
304. ##########################################
305. # CHANGE DIRECTORY           #
306. ##########################################
307. function chdir {
308.     verbose "Changing to directory: $@"
309.     cd $@  
310. } # err
312. ##########################################
313. # DEBUG FUNCTION             #
314. ##########################################
315. #
316. # Like 'echo', but prints only if debug operation is specified.
317. function dbg {
320.     [ "$_DEBUG" == 1 ] && echo "[DEBUG] $@"
321. } # dbg
323. ##########################################
324. # ERROR FUNCTION             #
325. ##########################################
326. #
327. # Like 'echo', but prints to error stream
328. function err {
329.     echo "[ERROR] $@" 1>&2
330. } # err
332. ##########################################
333. # VERBOSE FUNCTION           #
334. ##########################################
335. #
336. # Like 'echo', but prints only if verbose operation is specified.
337. function verbose {
338.     [ "$VERBOSE" == 1 ] && echo "$@"
339. } # verbose
341. ##########################################
342. # INSTALL ARGOWEB APPS           #
343. ##########################################
345. dbg "Running in debug mode..."
347. # Make sure user is root
348. if [ "$(id -u)" != "0" ]; then
349.    err "This script must be run as root"
350.    exit 1
351. fi
353. # Get command line arguments
354. while getopts ":cd:v" opt; do
355.     dbg "processing command-line argument: -$opt"
356.     case $opt in
357.         c)
358.             # clean working directory.
359.             rm -rf mysql tomcat
360.                 ;;
362.         d)
363.             # Specify working directory. This is the directory to which all files will be downloaded and unzipped.         
364.             src_dir=$OPTARG
365.             chdir $src_dir
366.                 ;;
368.             v)
369.             # Verbose operation
370.                 VERBOSE=1
371.             VOPT="-v"
372.             TAROPTS="-xzvf"
374.             verbose "Operating in verbose mode."
375.                 ;;
377.             \?)
378.                 err "Invalid option: -$opt"
379.             exit 1
380.                 ;;
382.         :)
383.                 err "Option -$OPTARG requires an argument."
384.                 exit 1
385.                 ;;
386.     esac
387. done
389. dbg "VERBOSE=$VERBOSE"
390. dbg "src_dir=$src_dir"
392. verbose "Working directory: $src_dir"
393. chdir $src_dir
395. # Prompt and install MySQL database
396. ask "Install MySQL database?" Y && install_mysql
398. # Prompt and install Apache Tomcat Server
399. ask "Install Apache Tomcat Server?" Y && install_tomcat