Changes between Version 95 and Version 96 of ServerIntro


Ignore:
Timestamp:
Sep 23, 2017, 4:35:22 AM (7 years ago)
Author:
Seth
Comment:

Clarified different username used by Apache on Ubuntu or Debian (www-data). The usermod command was wrong and had the flags in the incorrect place. This has been corrected.

Legend:

Unmodified
Added
Removed
Modified
  • ServerIntro

    v95 v96  
    22
    33= Setting up a BOINC server = #ServerIntro
    4 
    5 You can set up a BOINC server in a commercial could, such as
    6 [wiki:CloudServer Amazon] or Google.
    7 The hardware and most of the software issues are taken care of for you.
    8 You'll have to pay,
    9 but it may be cheaper than using your own hardware.
     4You can set up a BOINC server in a commercial could, such as [wiki:CloudServer Amazon] or Google. The hardware and most of the software issues are taken care of for you. You'll have to pay, but it may be cheaper than using your own hardware.
    105
    116If you choose to use your own hardware, there are several software options:
    127
    13  * Use [https://github.com/marius311/boinc-server-docker a set of Docker containers]
    14   developed by Marius Millea.
    15 
    16  * Use a [wiki:VmServer BOINC server VM] that we've created,
    17   with all the necessary software already installed.
    18   You can run this virtual machine using
    19   [https://www.virtualbox.org/ VirtualBox]
    20   on any Intel-based computer (Windows, Linux, or Mac OS X).
    21 
    22  * Debian and Ubuntu offer a "boinc-server-maker" package to create
    23    BOINC projects more easily on local or remote machines.
    24    This ongoing effort is described
    25    [http://wiki.debian.org/BOINC/ServerGuide here].
    26 
    27  * Build BOINC from source on your own system.
    28    You can use any Unix system as a BOINC server;
    29    we recommend using a recent Linux release.
    30    Disable SELinux. General instructions are given below,
    31    followed by a [wiki:ServerIntro#CookbookforDebian6.0 cookbook for setting up a BOINC server on a Debian 6.0 system].
     8 * Use [https://github.com/marius311/boinc-server-docker a set of Docker containers] developed by Marius Millea.
     9
     10 * Use a [wiki:VmServer BOINC server VM] that we've created, with all the necessary software already installed. You can run this virtual machine using [https://www.virtualbox.org/ VirtualBox] on any Intel-based computer (Windows, Linux, or Mac OS X).
     11
     12 * Debian and Ubuntu offer a "boinc-server-maker" package to create BOINC projects more easily on local or remote machines. This ongoing effort is described [http://wiki.debian.org/BOINC/ServerGuide here].
     13
     14 * Build BOINC from source on your own system. You can use any Unix system as a BOINC server; we recommend using a recent Linux release. Disable SELinux. General instructions are given below, followed by a [wiki:ServerIntro#CookbookforDebian6.0 cookbook for setting up a BOINC server on a Debian 6.0 system].
    3215
    3316== Hardware ==
    34 For experimentation and debugging,
    35 you can use almost any computer as a BOINC server.
    36 Before deploying a project more widely,
    37 make sure that your server has adequate performance, availability, and security.
    38 Some factors:
     17For experimentation and debugging, you can use almost any computer as a BOINC server. Before deploying a project more widely, make sure that your server has adequate performance, availability, and security. Some factors:
    3918
    4019 * Your server must have a static IP address.
    4120 * Your Internet connection should have adequate performance and reliability.
    42  * Your server should have 64-bit processors,
    43    at least 8 GB of RAM, and at least 40 GB of free disk space.
    44  * Do whatever you can to make it highly reliable (UPS power supply,
    45    RAID disk configuration, hot-swappable spares,
    46    temperature-controlled machine room, etc.).
    47  * Put it behind a firewall; allow access via port 80 (HTTP)
    48    and optionally 443 (HTTPS).
    49  * Make it secure; turn off any unneeded network services,
    50    especially those that use plaintext passwords (like FTP or Telnet).
     21 * Your server should have 64-bit processors, at least 8 GB of RAM, and at least 40 GB of free disk space.
     22 * Do whatever you can to make it highly reliable (UPS power supply, RAID disk configuration, hot-swappable spares, temperature-controlled machine room, etc.).
     23 * Put it behind a firewall; allow access via port 80 (HTTP) and optionally 443 (HTTPS).
     24 * Make it secure; turn off any unneeded network services, especially those that use plaintext passwords (like FTP or Telnet).
    5125
    5226Info on increasing capacity and reliability is [wiki:MultiHost here].
     
    5428== Installing the BOINC server on Unix == #general
    5529=== Groups and permissions === #permissions
    56 
    5730BOINC server programs run as two different users:
    5831
    59  * The scheduler, file upload handler, and web software
    60    run under the web server account
    61    (on Fedora this is user '`apache`'; on Debian it's '`www-data`').
    62 
    63  * Other programs run under a normal user account,
    64    called the "project owner".
    65 
    66 The project owner may be your existing account,
    67 or you can create a new account for this purpose with '''useradd'''.
    68 In the following we'll assume that the project owner
    69 is '''boincadm''', with primary group '''boincadm'''.
     32 * The scheduler, file upload handler, and web software run under the web server account (on Fedora this is user '`apache`'; on Debian it's '`www-data`').
     33
     34 * Other programs run under a normal user account, called the "project owner".
     35
     36The project owner may be your existing account, or you can create a new account for this purpose with '''useradd'''. In the following we'll assume that the project owner is '''boincadm''', with primary group '''boincadm'''.
    7037
    7138'''Do not use root as the project owner'''.
    7239
    73 By default, the directories created by user `apache` are not world-writable.
    74 This causes problems: for example, when the file upload handler creates
    75 a directory in the [wiki:DirHierarchy upload hierarchy],
    76 it's owned by (`apache`, `apache`),
    77 and the [wiki:FileDeleter file deleter]
    78 (which runs as `boincadm`) won't be able to delete the files there.
    79 
    80 To solve this problem, add `apache` to the to group `boincadm` using
    81 {{{
    82 $ usermod -G -a boincadm apache
    83 }}}
    84 
     40By default, the directories created by the apache web server user (on Fedora this is user '`apache`'; on Ubuntu or Debian it's '`www-data`') are not world-writable. This causes problems: for example, when the file upload handler creates a directory in the [wiki:DirHierarchy upload hierarchy], on Fedora it's owned by (`apache`, `apache`), and the [wiki:FileDeleter file deleter] (which runs as `boincadm`) won't be able to delete the files there.
     41
     42To solve this problem on Fedora, add `apache` to the to group `boincadm` using
     43
     44{{{
     45$ usermod -a -G boincadm apache
     46}}}
     47To solve this problem on Debian or Ubuntu, add `www-data` to the to group `boincadm` using
     48
     49{{{
     50$ usermod -a -G boincadm www-data
     51}}}
    8552or edit `/etc/group` and change the line
     53
    8654{{{
    8755boincadm:x:566:
    8856}}}
    8957to
     58
    9059{{{
    9160boincadm:x:566:apache
     
    9564Apache will need to be restarted for this to take effect.
    9665
    97 Both `boincadm` and `apache` should have
    98 [http://en.wikipedia.org/wiki/Umask umasks]
    99 that allow group read and write.
    100 The place to set this depends on what distribution is used.
    101 Normally you can use a umask of 0002 or 0007.
    102 
    103 When you create a BOINC project using [wiki:MakeProject make_project],
    104 the critical directories are owned by `boincadm` and have the set-GID bit set;
    105 this means that any directories or files created by `apache`
    106 in those directories will have group `boincadm` (not group `apache`).
    107 The BOINC software makes all directories group read/write.
    108 Thus, both `apache` and `boincadm` will have read/write access
    109 to all directories and files, but other users will have no access.
     66Both `boincadm` and `apache` or `www-data` (depending on your linux flavour) should have [http://en.wikipedia.org/wiki/Umask umasks] that allow group read and write. The place to set this depends on what distribution is used. Normally you can use a umask of 0002 or 0007.
     67
     68When you create a BOINC project using [wiki:MakeProject make_project], the critical directories are owned by `boincadm` and have the set-GID bit set; this means that any directories or files created by `apache` or  `www-data` in those directories will have group `boincadm` (not group `apache` or `www-data`). The BOINC software makes all directories group read/write. Thus, both `boincadm` and `apache`  or `www-data`  will have read/write access to all directories and files, but other users will have no access.
    11069
    11170To fix permissions on an existing project, do:
     
    11978chmod 02770 html/user_profile
    12079}}}
    121 You may also need to change the ownership of these directories
    122 and all their subdirectories to `boincadm/boinc`.
    123 If you're running several projects on the same server
    124 and want to isolate them from each other,
    125 you can create a different user and group for each project,
    126 and add `apache` to all of the groups.
    127 
    128 When serving your project files from Apache,
    129 note that all directories up to and including the `html` directory
    130 must have execute permissions.
    131 For example, if you use `make_project` to create the project template
    132 in your home directory,
    133 your home directory must have 711 permissions
    134 as opposed to the default of 700.
    135 If this is not corrected, you will receive a '''403 Forbidden''' error
    136 when attempted to browse to your project page.
    137 See [http://httpd.apache.org/docs/1.3/misc/FAQ.html#forbidden
    138 more information on dealing with Apache permissions problems].
     80You may also need to change the ownership of these directories and all their subdirectories to `boincadm/boinc`. If you're running several projects on the same server and want to isolate them from each other, you can create a different user and group for each project, and add `apache` or `www-data` to all of the groups.
     81
     82When serving your project files from Apache, note that all directories up to and including the `html` directory must have execute permissions. For example, if you use `make_project` to create the project template in your home directory, your home directory must have 711 permissions as opposed to the default of 700. If this is not corrected, you will receive a '''403 Forbidden''' error when attempted to browse to your project page. See [http://httpd.apache.org/docs/1.3/misc/FAQ.html#forbidden more information on dealing with Apache permissions problems].
    13983
    14084=== Installing BOINC software === #installing
    141  * Download and install the needed
    142    [wiki:SoftwarePrereqsUnix software prerequisites].
     85 * Download and install the needed [wiki:SoftwarePrereqsUnix software prerequisites].
    14386 * [wiki:SourceCodeGit Download] the BOINC software.
    14487 * [wiki:BuildSystem Configure and build] the BOINC software.
    14588
    146 === Make Project ===  #makeproject
    147 
     89=== Make Project === #makeproject
    14890After installation the BOINC software, you should run the [wiki:MakeProject make_project] script to create the project.
    14991
     
    15193
    15294=== Operating system configuration === #OS-config
    153 Some parts of the BOINC server (the feeder and scheduling server)
    154 use shared memory.
    155 Hosts where these run must have shared memory enabled,
    156 with a maximum segment size of at least 32 MB.
    157 How to do this depends on the operating system;
    158 see e.g. https://www.msi.umn.edu/~cpsosa/ChemApps/QuaChem/gaussian_tech/support/download/Linux_shared_mem_seg.html for more information.
     95Some parts of the BOINC server (the feeder and scheduling server) use shared memory. Hosts where these run must have shared memory enabled, with a maximum segment size of at least 32 MB. How to do this depends on the operating system; see e.g. https://www.msi.umn.edu/~cpsosa/ChemApps/QuaChem/gaussian_tech/support/download/Linux_shared_mem_seg.html for more information.
    15996
    16097=== MySQL notes === #MySQL-notes
    161 Instructions for setting up MySQL are here
    162 https://dev.mysql.com/doc/refman/5.6/en/linux-installation-yum-repo.html
    163 and here
    164 http://www.if-not-true-then-false.com/2010/install-mysql-on-fedora-centos-red-hat-rhel/
    165 
    166 After setting up MySQL,
    167 create a MySQL account for the project owner.
    168 We'll use the name '''boincadm''', same as the Unix account.
    169 All MySQL accounts should be password protected;
    170 we'll use "foobar" (use something different).
     98Instructions for setting up MySQL are here https://dev.mysql.com/doc/refman/5.6/en/linux-installation-yum-repo.html and here http://www.if-not-true-then-false.com/2010/install-mysql-on-fedora-centos-red-hat-rhel/
     99
     100After setting up MySQL, create a MySQL account for the project owner. We'll use the name '''boincadm''', same as the Unix account. All MySQL accounts should be password protected; we'll use "foobar" (use something different).
     101
    171102{{{
    172103mysql -u root -p
     
    174105GRANT ALL ON *.* TO 'boincadm'@'localhost';
    175106}}}
    176 This gives boincadm access to all MySQL databases on the host;
    177 if you use MySQL for other purposes on the host,
    178 set permissions more restrictively.
     107This gives boincadm access to all MySQL databases on the host; if you use MySQL for other purposes on the host, set permissions more restrictively.
    179108
    180109Other notes:
    181  * Set boincadm's `PATH` environment variable to include MySQL programs
    182   (typically `/usr/local/mysql` and `/usr/local/mysql/bin`).
    183  * You'll want to back up your database periodically.
    184   Generally this requires stopping the project,
    185   making a copy or snapshot, and restarting.
    186   An example is [//mysql_backup.txt here].
    187  * BOINC gets MySQL compiler and linker flags from a program called
    188    `mysql_config` which comes with your MySQL distribution.
    189    This sometimes references libraries that are not part of your base system
    190    installation, such as `-lnsl` or `-lnss_files`.
    191    You may need to install additional packages
    192    (often you can use something called 'mysql-dev' or 'mysql-devel')
    193    or fiddle with Makefiles.
    194  * MySQL can be the bottleneck in a BOINC server.
    195    To optimize its performance,
    196    read about [wiki:MysqlConfig configuring MySQL for BOINC].
     110
     111 * Set boincadm's `PATH` environment variable to include MySQL programs (typically `/usr/local/mysql` and `/usr/local/mysql/bin`).
     112 * You'll want to back up your database periodically. Generally this requires stopping the project, making a copy or snapshot, and restarting. An example is [//mysql_backup.txt here].
     113 * BOINC gets MySQL compiler and linker flags from a program called `mysql_config` which comes with your MySQL distribution. This sometimes references libraries that are not part of your base system installation, such as `-lnsl` or `-lnss_files`. You may need to install additional packages (often you can use something called 'mysql-dev' or 'mysql-devel') or fiddle with Makefiles.
     114 * MySQL can be the bottleneck in a BOINC server. To optimize its performance, read about [wiki:MysqlConfig configuring MySQL for BOINC].
    197115 * [//mysql_cluster.txt Notes on running MySQL on a cluster].
    198116
     
    202120
    203121=== Apache notes === #apache-notes
    204 In `httpd.conf` or `apache2.conf` on Ubuntu, set the default MIME type as follows
    205 (otherwise you'll get file upload signature verification errors):
     122In `httpd.conf` or `apache2.conf` on Ubuntu, set the default MIME type as follows (otherwise you'll get file upload signature verification errors):
    206123
    207124{{{
    208125DefaultType application/octet-stream
    209126}}}
    210 To limit denial-of-service attacks, we recommend turning off directory
    211 indexing by adding `-Indexes` to the
    212 [http://httpd.apache.org/docs/2.0/mod/core.html#options Options] directive.
    213 
    214 Make sure that the filename extensions used by your application or data files
    215 aren't handled in undesired way by Apache.
    216 For example, if some of your files have a '.map' extension, remove the line
     127To limit denial-of-service attacks, we recommend turning off directory indexing by adding `-Indexes` to the [http://httpd.apache.org/docs/2.0/mod/core.html#options Options] directive.
     128
     129Make sure that the filename extensions used by your application or data files aren't handled in undesired way by Apache. For example, if some of your files have a '.map' extension, remove the line
    217130
    218131{{{
     
    221134from httpd.conf
    222135
    223 Apache has a default request size limit of 1 MB.
    224 If your project is likely to exceed this
    225 (large output files, large scheduler request messages) increase this,
    226 e.g. to 128 MB:
     136Apache has a default request size limit of 1 MB. If your project is likely to exceed this (large output files, large scheduler request messages) increase this, e.g. to 128 MB:
    227137
    228138{{{
     
    230140LimitRequestBody 134217728
    231141}}}
    232 
    233142On Debian and Ubuntu Apache's cgi module is not enabled by default. Use the following command to enable it:
     143
    234144{{{
    235145sudo a2enmod cgi
    236146}}}
    237 
    238147=== PHP configuration ===
    239 
    240 You may need to add the GD and XML libraries to your PHP;
    241 You can do this with a command like
     148You may need to add the GD and XML libraries to your PHP; You can do this with a command like
     149
    242150{{{
    243151yum install php-gd php-xml
     
    245153then restart Apache.
    246154
    247 For more information:
    248 http://www.php.net/manual/en/image.installation.php
     155For more information: http://www.php.net/manual/en/image.installation.php
    249156
    250157If you build PHP from source, use the following:
     158
    251159{{{
    252160configure --with-jpeg-dir --with-png-dir --with-mysqli --with-curl --with-gd --with-zlib
    253161}}}
    254 
    255162=== PHPMailer === #PHPMailer
    256 
    257 By default, BOINC uses PHP's `mail` function to send email to participants.
    258 This uses sendmail.
    259 If this doesn't work, you can use
    260 [https://github.com/PHPMailer/PHPMailer PHPMailer],
    261 is a very flexible mail-sending mechanism, instead.
    262 To do this:
    263   * Download PHPMailer and put it under PROJECT/html/inc/phpmailer
    264     (i.e. the files class.smtp.php and class.phpmailer.php should be in that directory).
    265   * Edit your html/project/project.inc to add a function like
     163By default, BOINC uses PHP's `mail` function to send email to participants. This uses sendmail. If this doesn't work, you can use [https://github.com/PHPMailer/PHPMailer PHPMailer], is a very flexible mail-sending mechanism, instead. To do this:
     164
     165 * Download PHPMailer and put it under PROJECT/html/inc/phpmailer (i.e. the files class.smtp.php and class.phpmailer.php should be in that directory).
     166 * Edit your html/project/project.inc to add a function like
    266167{{{
    267168function make_php_mailer() {
     
    282183
    283184=== Common problems === #common-problems
    284 After adding a new user, you have to check if the home directory was created.
    285 Sometimes you have to manually create it.
     185After adding a new user, you have to check if the home directory was created. Sometimes you have to manually create it.
    286186
    287187{{{
     
    289189$ chown {username}:{username} /home/{username}
    290190}}}
    291 The link /usr/bin/gcc that points to the currently used compiler
    292 could be missing.
    293 This could lead to errors when running the make command.
    294 Especially when trying to compile the graphical part of the API
    295 you need the /usr/bin/g++ link
    296 (make gives a warning that it couldn't find GL/GLU/GLUT
    297 although it is installed).
    298 So you have to create the symbolic links,
    299 depending on your gcc and g++ version, first.
     191The link /usr/bin/gcc that points to the currently used compiler could be missing. This could lead to errors when running the make command. Especially when trying to compile the graphical part of the API you need the /usr/bin/g++ link (make gives a warning that it couldn't find GL/GLU/GLUT although it is installed). So you have to create the symbolic links, depending on your gcc and g++ version, first.
    300192
    301193{{{
     
    303195$ ln -s /usr/bin/g++-4.11 /usr/bin/g++
    304196}}}
    305 At newer linux distributions you can’t login as user root.
    306 Instead there is a user notroot with whom you can login,
    307 but you won’t have root privileges.
    308 If an application or script needs root privileges,
    309 you are forced to enter the password you entered during the installation.
    310 In the console window you can get root privileges with the
    311 su (Debian) or sudo (Ubuntu, Fedora) command also.
    312 Note the single quotes around the command when using su -c !
     197At newer linux distributions you can’t login as user root. Instead there is a user notroot with whom you can login, but you won’t have root privileges. If an application or script needs root privileges, you are forced to enter the password you entered during the installation. In the console window you can get root privileges with the su (Debian) or sudo (Ubuntu, Fedora) command also. Note the single quotes around the command when using su -c !
    313198
    314199{{{
     
    316201$ sudo {command}
    317202}}}
    318 If you have problems executing the sudo command,
    319 you have to add the user to the /etc/sudoers file.
    320 To edit this file just use this command:
     203If you have problems executing the sudo command, you have to add the user to the /etc/sudoers file. To edit this file just use this command:
    321204
    322205{{{
    323206$ sudo visudo
    324207}}}
    325 In this file you have to add another line after the line for notroot.
    326 You can use this:
     208In this file you have to add another line after the line for notroot. You can use this:
    327209
    328210{{{
     
    330212}}}
    331213== Cookbook for Debian 6.0 ==
    332 Please follow these recipes only for your basic orientation.
    333 The Debian landscape has changed a lot over since Debian 4.0.
    334 You can inspect the build and runtime dependencies of the
    335 boinc-server-maker package
    336 [http://anonscm.debian.org/gitweb/?p=pkg-boinc/boinc.git;a=blob_plain;f=debian/control;hb=HEAD here]
    337 and the latter more graphically also
    338 [http://packages.debian.org/unstable/net/boinc-server-maker here].
     214Please follow these recipes only for your basic orientation. The Debian landscape has changed a lot over since Debian 4.0. You can inspect the build and runtime dependencies of the boinc-server-maker package [http://anonscm.debian.org/gitweb/?p=pkg-boinc/boinc.git;a=blob_plain;f=debian/control;hb=HEAD here] and the latter more graphically also [http://packages.debian.org/unstable/net/boinc-server-maker here].
     215
    339216=== Install software prerequisites === #cookbook-prereq
    340 Common packages for building BOINC (based on
    341 [wiki:SoftwarePrereqsUnix Software prerequisites (Unix/Linux)]):
     217Common packages for building BOINC (based on [wiki:SoftwarePrereqsUnix Software prerequisites (Unix/Linux)]):
    342218
    343219{{{
     
    382258libsqlite3-dev
    383259}}}
    384 
    385260optional packages for a graphical desktop manager
    386261
     
    392267xserver-org
    393268}}}
    394 This list contains all necessary packages for the BOINC server and the client
    395 components.
    396 At first startup, these packages should be installed with aptitude.
    397 The graphical desktop manager is optional,
    398 but makes the configuration of the server a bit easier.
    399 The list contains no security packages to secure your server!
    400 Please ask your system administrator or read a good Linux book
    401 on how to secure your server against attacks.
     269This list contains all necessary packages for the BOINC server and the client components. At first startup, these packages should be installed with aptitude. The graphical desktop manager is optional, but makes the configuration of the server a bit easier. The list contains no security packages to secure your server! Please ask your system administrator or read a good Linux book on how to secure your server against attacks.
    402270
    403271Hint: You can search packages within aptitude by pressing the '/' key.
    404272
    405273=== Create a new BOINC user account === #cookbook-useraccount
    406 The best way to create the boinc user is during installation
    407 when you are asked for name and password of the main user.
    408 Enter 'boincadm', or a name of your choice, as login name.
    409 In this cookbook I will refer to this useraccount as 'boincadm'.
    410 
    411 When installation is finished please login as the user created during installation.
    412 If you have installed a graphical desktop manager please open a text console (Konsole or xterm).
     274The best way to create the boinc user is during installation when you are asked for name and password of the main user. Enter 'boincadm', or a name of your choice, as login name. In this cookbook I will refer to this useraccount as 'boincadm'.
     275
     276When installation is finished please login as the user created during installation. If you have installed a graphical desktop manager please open a text console (Konsole or xterm).
    413277
    414278Next step is to add the 'www-data' user (apache webserver) to the group of the newly created user:
     279
    415280{{{
    416281$ usermod -a -G boincadm www-data
    417282}}}
    418 
    419283=== Configuration of the MySQL server === #cookbook-mysql
    420284Create a new MySQL database user account.
     
    425289> SET PASSWORD FOR 'boincadm'@'localhost'='';
    426290}}}
    427 The permissions can be limited to project database only, later;
    428 Defining an empty password simplifies the installation, can be changed later.
     291The permissions can be limited to project database only, later; Defining an empty password simplifies the installation, can be changed later.
    429292
    430293You can also set this using the phpmyadmin interface available through !http://{server-IP}/phpmyadmin
     
    443306$ git pull
    444307}}}
    445 
    446 A stable '''client''' version can be obtained from [//download_all.php BOINC website].
    447 Hint: Normally you won't need a self-compiled client,
     308A stable '''client''' version can be obtained from [//download_all.php BOINC website]. Hint: Normally you won't need a self-compiled client,
    448309
    449310NOTE: Tags are only for client software, always checkout the latest version for the server!