LabPrimer
Best practices for pocket research and development labs in informatics. Integrates agile industrial practices with ethnomethodology, interaction design, sense-making and more.
|
|
|
Guests are welcome to view our materials. To subscribe, edit, view raw markup, etc., you'll need to register for an account. Accounts are free (and will always be free) - your involvement helps us directly and indirectly (by demonstrating that our work matters to our funders...) StartingPoints has more info.
LabPrimer
ServerAdminGuidelines is an all-in-one presentation of the topics covered in the LabPrimer project. The topics covered here can also be accessed individually from the links on the left navigation bar.
Please feel free to join our effort and contribute your thoughts, ideas, experience and questions. See the Content Model topic (below) for an explanation of how we organize topics. Anyone can register for an account and get involved.
Infrastructure Design Philosophy[edit] Establishing and maintaining a successful academic computing research lab requires a different infrastructure design than those used for industrial computing research labs or academic computing teaching labs. The approach that has worked for us is an agile approach best described as managed chaos. Our infrastructure design prioritizes:
Laziness, Impatience, Hubris, Diligence, Patience and Humility[edit] One of the most difficult balancing acts for an academic researcher running a research lab is striking the right balance between (1) getting things done, (2) professional development, and (3) materials development. In Diligence, Patience, and Humility, Larry Wall expands his well-known discussion of the three virtues of a programmer to three virtues of an agile (my word, not his) community. The key is, as Larry says, to recognize that these seemly opposing needs can actually be met at the same time. Tools like this wiki are a crucial part of the process. Using them in an agile manner is another crucial part of the process. Letting go of the need to be perfect is yet another - you have to develop the willingness to put interim versions of your work out there for others to see. And be ready for some people to not understand and be fairly nasty about it. Once you develop the willingness, then everything else follows. Instead of just doing things the way you already know how to do them, always take some time to explore alternatives. Track however far you get on those alternatives in the wiki. When you get a chance, you can keep going. In the meantime, someone else may pick up where you left off. No matter what, folks who are newer in the lab (or in another lab somewhere else) will learn from your notes and thought process. Everybody wins. You loose the vast majority of the benefit of exploring alternatives, though, if you don't put your explorations out somewhere public, because computing systems move too fast for the individual 30 minutes to 3 hour chunks of time you get to add up without community. -- HilaryHolz - 10 May 2008 back to topWork and play well with others[edit] Our labs live in a larger university environment. What's more, we tend to comprise a very small portion of that university computing environment. Nonetheless, so-called 'pocket labs' have historically been responsible for many of the worst university computing catastrophes. Our relationship with university IT is truly collaborative (Hilary sits on the UIT committee.) I (Hilary) believe that we've been able to evolve that relationship based on mutual respect and an excellent lab security record. In general, the lab follows the Center for Internet Security Standards. Details in various places. -- HilaryHolz - 10 May 2008 back to topLess is more[edit] It's tempting to think that a lab machine is just a research machine, so it isn't really critical to worry about issues of security, optimizing performance, etc. However, lab machines are favorite targets of hackers precisely because that mindset is so common. Leave aside concerns about loss of your precious work for a moment. You would be astonished what portion of a machine's resources are wasted simply by fending off attacks using the default profile of a distribution - that is what a denial of service attack is, after all. Also, you'll also be (pleasantly) astonished at the speedup resulting from disabling (or even better, removing) all the bloatware that even linux now seems to think we all just cannot do without. (Do I really need 4 or 5 competing solutions for wireless connectivity on my server, which has no wireless card? Fellas, I'll download it if/when I need it...) -- HilaryHolz - 10 May 2008 Hanging on to all that extra stuff that you aren't using wastes your resources in every sense of the word
A Sense-Making Content Model[edit] This topic presents our evolving ContentModel, integrated with a number of best practice recommendations. The content model is grounded in Sense-Making. The current presentation reflects the Sense-Making foundation and the fact that I haven't had a chance to rewrite this description.How we got here (methodology)Initially, we reviewed the existing pages in our TWiki as well as several other TWikis, held discussions among our existing membership, reviewed our own journal entries, and proposed a simpler model (which can be viewed in earlier revisions of this topic.) After living with that model for a while, we conducted some contextual interviews with members, reviewed entries from the initial content model, and refined the model to this model. One thing that is clear is that those community members who have been most aware of the earlier model have been most successful in their use of the TWiki. AhatSkin is still fairly young at this time, but we hope that it will help in promoting visibility of the model.Joining theory and practice (MetaTheory)Authorship (Ideology)Authorship of topics lies on a spectrum ranging from
Types of pages (Epistemology)
Designing Information over time (aka Digital Inclusion aka Ontology)[... add a quick explanation of how digital inclusion relates to issues in access to understanding agile methods and the free/libre software technologies used in TWiki ...] some specific refactoring recommendations
Managing Unix[edit] having problems? see DontPanicneed to RtFM? We can help The Ahat lab, which hosts this wiki, uses a rich set of platforms. Our professional work, however, tends towards a blend of Unixes. The big servers are all on Fedora, mostly on core 9. We tend towards bleeding edge webserver/language/interaction research and development rather than Unix kernel development itself, so walking the knife-edge of being on the most recent truly stable Fedora core (i.e., lagging one "production" core behind) has given us the best trade-off of bleeding edge agility vs. bleeding edge instability. We've played with Debian (and some others), but find ourselves back with Fedora. It's not perfect, but it's home... Personally, many of us use Darwin, Apple's BSD-style Unix OS, as Mac laptops are the laptop of choice (for those with the funding to afford them.) Folks with windows laptops tend to start by installing Ubuntu on their laptops. As a starter Linux, Ubuntu is the clear current winner, and it's an easy knowledge migration from Ubuntu to Fedora. When we need to test things on Windows, we tend to rely on VirtualizatioN. It's a digital inclusion thing; we find that the payoff from putting our limited resources into FOSS has a better return. Fedora is the free software project companion to Red Hat Linux, Enterprise edition. The information here tends towards fedora, largely due to the tools available for managing servers for scientific programming. We would welcome a companion set of materials about how to achieve the same or similar effects with other platforms. Installing Fedora[edit] There are a rich variety of ways to install Fedora (as well as many other distributions of Linux.) Essentially they all break down to the following steps:
Installing Fedora 10 via Live CD[edit] One of the ways to install some Linux distributions is via a live CD. A live CD is a bootable CD from which you can install the distribution. Why install from a live CD? It comes with less crud. But it is harder to install. See Fedora install guide for f10.
Managing and distributing packages[edit] Software is distributed on Linux distributions in packages managed by tools appropriately called package managers. Once upon a time those package managers were also used to distribute the packages (ask an old-timer abouturpmi some day and enjoy the rant), but now we have tools to manage distributing packages rather than the packages themselves sometimes called updaters. For example, the Red Hat package manager (RPM) is a free software package manager used by Red Hat Enterprise, Fedora Core, SuSE, and Mandrake distributions, among others. A variety of updaters are available for packages produced by RPM (.rpm files) including yum (command line), pirut (GUI) and MaintainingDistributionsWithPackageKit GUI.
Package managers and updaters play a critical role in CollaborativeToolBuilding. For end-users, these tools automate the process of letting you know when updates of the software you use are available, installing those updates, etc. For developers, these tools serve a much more important role; they help us expose and track the relations between our work, refining and refactoring those as we go.
What's in a package?Package contents vary a lot by Linux distribution, type of package, and best practices of the community to which the developers belong (not to mention the degree to which the developers adhere to those best practices!) A package includes (in a perfect world) everything needed to install, remove, and upgrade the software in question on the distribution in question. More specifically, any one package may include:
.rpm files, rpm --filesbypackage will tell you what files will be installed by a package, but not what scripts are included in a package.
Maintaining distributions with Yum[edit] Yum (yellowdog updater modified) is the update layer for distributions using RPM packages (see also PackageManagers for a general discussion of package managers vs. updaters.) The main configuration file foryum (on fedora) is /etc/yum.conf. To add something to the excludes list (or to start one), see this example YumDotConf. Updates should be run regularly (whether nightly or weekly), although probably not via a cron job.
In general, use yum in upgrade mode rather than update mode, so as not to leave older versions of packages lying about, as this causes real problems. You can also use package-cleanup (see below.)
In addition to yum itself, there are the yum-utils (separate package), and a variety of useful plugins. Make sure you enable plugins in /etc/yum.conf.
Maintaining distributions with PackageKit[edit] PackageKit is a cross-distribution, cross-architecture updater. There's a yum interface to PackageKit and a command line tool calledpkcon so you don't have to dork around learning yet another slow, transient GUI tool. At the date of this update (-- HilaryHolz - 29 Mar 2009), although the situation had improved over the last six months, PackageKit remains too new (version 0.3.14) and sketchy to be worth the work in all:
Upgrading Fedora With Yum[edit] First, a quick comment on why? Why do a major (core) upgrade of your workstations and/or servers with yum (or analog), rather than starting with a clean install each time? Gee, I would have thought that was obvious, in all honesty. We've got our servers configured and tuned perfectly for our needs. A major upgrade comes out. That upgrade is, of necessity, one-size-fits-all. Telling us to start all over is, in essence, telling us that someone (or some team) who has never met us, never seen our situation, knows literally nothing about our particular needs knows better than us. To get back to where we were will take some amount of looking through what has changed. I would much, much rather assume that we know what we need, and phase in the new configuration, than assume that someone else knows what we need and load a whole bunch of what is almost surely dancing bear ware on our servers. I am more than happy to serve as a beta deployment platform for certain select packages, but not for all and sundry, including gnuchess (yes, that's GNU Chess). So I'm conservative. I install the base package and the packages we know we need, and leave out the rest until they come to my attention. Some general comments:
Removing Extraneous Packages Will Keep Your Server Happy and Healthy![edit] In the natural course of things, linux servers accumulate both orphans (packages installed fromrpms that are not associated with any current repository) and cruft (packages that are not relevant to the mission of the server.) Periodically reviewing and, in most cases, removing, such packages is a good thing (tm). Seriously, though, a review will:
OrphansWhat is an Orphan?Technically, anorphan (yum calls these extras) is a package installed from an rpm that is not associated with any current repository.
What should I do with Orphans?That depends on the type oforphan:
Finding OrphansDepending on which version of which Linux distribution you are using, you may have one or more of the following tools available.
CruftCruft, like orphans, comes in several flavors. Many kinds of cruft can actually be prevented from accumulating by careful configuration ofyum.
Preventing CruftConfiguring yum to prevent cruft (would love to have similar info contributed forapt-get, etc...)
LeavesLeaves are packages installed fromrpms that are not depended upon by any other rpms. While being a leaf does not make a package cruft, leaf packages are likely candidates to be cruft, particularly if those leaves are libraries. package-cleanup, in yum-utils, is particularly helpful in identifying leaves that are cruft.
Dancing Bear WareReview all the packages that have accumulated on your system:
yum erase < packagename > and see if the results are terrifying.
Understanding DependenciesThe dependency system is at the heart of the collaborative development of FOSS. PackageManagers provide a relatively easy way for package developers to specify which package(s) their package depends on. The package manager generates dependency chains from this dependency information, resolving dependency issues as it progresses. Pay attention to dependency information while removing packages, or you may very easily end up with a dead or malfunctioning server. You may have to override dependency info from time to time (see Circular dependencies), so here are some general guidelines:
Circular dependenciesCircular dependencies occur when package A depends on package B, but package B depends on package A. You can have a longer loop as well (package A depends on Package B depends on package C which depends on package A). You can remove circular dependencies withrpm --erase --nodeps < packagename >, but you must remove all the packages in the circle.
Removing Packages with Yum
 (don't use yum -y here!). If yum wants to delete all sorts of stuff for dependencies, look carefully. It doesn't hurt to google the package and any error messages you may be getting with yum erase or yum remove before continuing. Sometimes, it's ok. For example, you may be removing orphans, and what it wants to delete may also be orphans. If not, you may want to let yum delete the packages and then re-install, or you may want to fall back to rpm
Removing Packages with RPMYou should only attempt to userpm to erase packages if yum can not remove the package due to various errors. Do NOT use this command lightly!
You are on your own if you get some wild ideas to just yank packages out with no regard for dependencies.
What if I've removed too much?
Perl's package managers: cpan and cpanp[edit] Perl has its own package managers (see BootstrappingPerl), which should be used in preference to rpm or yum. There are two interfaces (modules) in use right now - a lower layer of abstraction, the CPAN module (MCPAN) and a higher layer of abstraction, the CPANPLUS module (MCPANPLUS) to which the community is migrating, slowly. It's part of a larger migration to 'pure perl' based on a new module abstraction layer (Module::Build). In any case, you should use cpanp for most things, however, for some things you will still need to fall back to perl -MCPAN -e shell. Some hints about using cpanp.
Managing Security[edit] Security is a huge subject. The information we provide here is based on our experience, and you follow it at your own risk, we make no promises, warranties, etc., implied or otherwise. The Center for Internet Security publishes best practice guidelines tailored to all the major platforms as well as many important services: apache, various SQL database servers, etc. The Internet Storm Center collects and disseminates information about internet security threats on an ongoing basis, as well as providing useful tools for combating a variety of threats. See also:
Practical, usable securitySecurity isn't secure if you won't use it! The initial configuration sequence for our minimally reasonable, responsible, and usable configuration is given here. Note that all the instructions are command-line, as GUI interfaces are much slower to use, and not generally appropriate for servers.
Use Sudo![edit]sudo is a system that (1) provides you with a log of everything you do as superuser, and (2) allows finer grained control over who can do what than simply putting trusted users into group wheel. Really, the only time you should ever use the su command to act as root is to setup sudo.
Getting started with sudoWe provide a crash course in setting upsudo , however, don't forget to RtFM as well, as it is your security that is on the line, not ours!!
The configuration file for sudo is /etc/sudoers. You have to use a special command, visudo, to edit /etc/sudoers . You can make visudo use your favorite editor (rather than vi ) by setting the environment variable EDITOR in your shell before invoking visudo , but that's properly the subject of another topic.
Better use of sudoIt's really better not to just put folks in wheel and let them have full privileges. One of the wonderful things aboutsudo is it's ability to confer heightened privileges within a limited scope. So you can give privileges to mount and unmount media, but not other things, or to restart certain servers, for example.
back to top
denyhosts[edit] denyhosts is a python daemon that detects and thwarts brute-force ssh attacks. The configuration script, by the way, needs to be /etc/denyhosts.conf, but seems to be installed as /etc/denyhosts.cfg in some versions. Just read and follow the comments in the configuration file. back to topPorts & iptables[edit] On any new server, you'll need to ask to have any ports you want accessible from off campus explicitly entered into the university's firewall setup. Send email to the designated contact (this should really be a link to the appropriate entry in the IT directory, you know..., currently, for us, that's richard.uhler@csueastbayHYRUQAPZ.edu) with the request. You'll need to explain what ports you want open, and why. One way to learn what we usually have open is to look at a sample IpTables configuration file with lots of comments and figure out what services apply to your server. Always ask for ports 50000 - 50500, as those are the Math and Computer Science research ports - ports on which we are free to take services up and down at will, but are not preallocated to anything in particular. If you reserve a particular port on a particular server for some long term use, you should add that info to the relevant entry in the LabData topic. In combination with good security practices set out by the CIS guidelines (as well as those discussed in this twiki) such as using DenyHosts, ModSecurity, SpamAssassin, ..., having some externally open ports, even the research ports, has not proven to be a major issue. Speaking of iptables (a firewall that runs on linux servers), we run iptables on all our machines. Don't open any ports unless you know what/why, even if a program wants you to do so. Ask for help. /etc/services lists all the port numbers with an incredibly terse description of the related service, but it will give you a key word to Google for. back to topsshd, the secure shell daemon[edit] The ISC guidelines cover most of the important ssh configuration information. We provide some more detail to help you get going, as well as explain how to prevent remote connections from getting dropped all the time!Configuring sshPassword authentication is inherently unsafe, not to mention a pain-in-the-you-know-what. Other forms of authentication, such as public key or certificate authentication may be a much better fit, depending on your lab setup. The configuration we show here is for public key authentication (no passwords), with PAM (pluggable authentication modules) account and session checks. We show only thesshd_config settings we change from the default settings, and group them a bit differently than the order they appear in the file to make it easier to explain.
# HJH, duh! PermitRootLogin no # HJH, per CIS_RHEL_Benchmark IgnoreUserKnownHosts yes # HJH, no password authentication PasswordAuthentication no # HJH, disable s/key passwords ChallengeResponseAuthentication no # HJH, enable PAM session processing UsePAM yes # HJH, poll client every 30 secs, so router doesn't # drop connection. Settings give 25 minutes of inactivity. ClientAliveInterval 30 ClientAliveCountMax 50 # HJH, as per CIS_RHEL5_Benchmark Banner /etc/issue.netBe sure to review the pam related packages to be sure that you have the appropriate ones installed on your system. We'd love to give you a canonical list, but they get re-factored all the time.
Configuring public-key authenticationThe specifics vary somewhat according to the specific protocol, but the general outline is:
Death By Networking[edit] Pocket labs just don't fit in to the networking scheme of larger IT. With Cyberinfrastructure all the rage (there's a vaporware term, goes right along with Computational Thinking, another of my favorites), we have a very real, very growing need to come up with better systems administration turnkey solutions to minimize our impact on our host institutions. That's a focus area for me, and one of the reasons I'm leaving academia for industry (because it seems to be a necessary step for me to be able to work on the problem....) In any case, for the moment, here's our older manual setup, with more automated info to follow the moment we have some breathing space to document what we're using:DHCP and pocket research lab servers[edit] DHCP (dynamic host configuration protocol) is a system that helps with efficient and secure management of a network of computers. The networking information is kept on a central server (the DHCP server), and the individual computers run DHC client programs which contact the DHCP servers on startup. This plan works fine when either (a) all the machines are administered by some central organization (the case for computers in classroom laboratories, for example, which are administered by university IT), or (b) the DHCP server and the client computers are fully decentralized (the case with faculty and student laptops, for example.) The system breaks down, however, for pocket research lab servers, which are, by necessity, neither fully integrated into the UIT structure, nor always easily accessible by research staff. For example, in an unplanned university-wide power outage, pocket research lab servers reboot before the DNS servers, end up with incorrect network information, and become unreachable, unless each one has its own UPS. Hardware that provides the ability to remotely power cycle the servers is available, but costs even more than a UPS. A much cheaper answer is to collaborate with UIT on an initial configuration for each server via DHCP, and then convert the server to a static configuration, documenting the relationship to the dynamic configuration meticulously in the relevant configuration files on the server itself. That way, desired changes to the networking setup can be coordinated with UIT, the servers are robust with respect to the vagaries of life in the big city, the local control necessary for research and learning is retained, and we save a bunch of $$$$. A perfect example of a situation in which the fix becomes obvious when you view the mediating artifact as an integrated whole, rather than focusing exclusively on the programmable components. Speaking of focusing on the programmable components, the situation is further complicated by the fact that most Unix distributions are just sure that you want to run DHCP, and will revert you back to that default configuration at every major upgrade, unless you take explicit action to prevent that from happening. So be sure to add dhclient and *dhcp* to the excludes list in YumDotConf once you have converted to a static networking setup. In time we'll get more surgical about just what needs to be excluded. back to topstatic networking setup[edit] (If you have not read DHCPornotDHCP, read that first.) In addition to erasing the dhclient and various dhcp packages, you should also erase NetworkManager. Manually configure:
Managing data[edit] see also ManagingConfigsQuick Start
Pocket Labs and Mirrored RepositoriesAs a research lab, it's important to not have the same files on all the servers in your account, but to have the option to have different files on different machines. It's also important for us to be able to share code on many of our projects. We have separated the abstractions of authentication and authorization from the abstraction of a file system, and are using a state-of-the-art answer to the question of file system. (We'll get back to you about state-of-the-art answers to authentication and authorization. ) We use subversion (online book, project home) + SVK (online book, project home), a distributed authoring and versioning file system, aka mirrored repository, aka DAV_FS. Subversion is the versioned file system (repository), while SVK is the distributed (mirrored) component of that technical mumbo-jumbo. Because we're a research lab and our needs are less structured and predictable than an academic department or an ISP, you essentially organize your files as you please, checking them out and in to whatever machine you are working on at the moment. Data in the versioned file system is organized (conceptually) into projects. Projects can be shared or private, depending on the authorization settings. What's more you can mirror part or all of a repository on a machine with SVK, so you don't need network access to check your changes in. You can synchronize with the central repository when you get network access. Once you have your own private project, you don't need any further authorization to create subprojects, etc., that are private.Getting started with SubversionIf you've never used a version control system, you'll want to read (at least) Fundamental Concepts & Basic Usage. Don't panic, though, they are quick reads. If you have used a version control system such as RCS, or an older distributed authoring systems such as CVS, you can get started with a quick skim of Basic Usage. You're cheating yourself, however, if you don't read more. Subversion is much, much more than simply friendlier CVS or RCS for groups. When working with group project files and/or server configuration project files, you'll also need to read branching and merging.Getting started with SVKThe subversion documentation is less mature than the subversion documentation, so the best approach is to go through the subversion docs, then our SVKCheatSheetBackupsMirroring provides backup capability as well as version control, so long as all the important information is kept in the mirrored file system. That leaves the question of backing up the repository itself, which subversion and SVK both support.A mirror case studyThe online subversion book gives all sorts of great info about using a readonly mirror withsvnsync as a continuous backup of all your data. Actually doing the configuration and setup is left as an exercise for the reader, as it were.
back to top
Managing Server Configuration Files[edit] Any server configuration file that is not 'boilerplate' should live in the serverfiles project in the lab's file system. When you want to change a configuration file, check first to see if the configuration file is part of the serverfiles project for the server in question. If no (or adding a new system to an Ahatlab server), please add the file to the appropriate project and update the LabData topic. When altering configuration files, for each change, comment out the line with the default value, make a copy of the line with the default value immediately below the commented out line and make changes to the copy. Initial and date the change. This process helps a lot with retaining and sharing knowledge - as researchers, we tend to delve into a server or OS topic related to our interests, eventually developing a lot of expertise in an area, then not needing that expertise for weeks, months, or even years. It's frustrating to have to relocate and/or relearn information later. So document things at the moment you do them. The specific process described also optimizes tracking changes with diff reports when doing version upgrades, whether of a specific subsystem or an entire OS (see UpgradingFedoraWithYum.) Puppet and Cft combine to form a much more agile approach to configuration management.Using Puppet Configuration Recipes[edit] Puppet is a declarative language designed to
64 bit systems[edit] On a 64bit machine, you occasionally need both the x86_64 packages and the i386 (i.e., 32 bit) packages. As best we currently understand it (folks who are more into this, please pitch in, or at least dig up good references for more reading and link those in), the difficulty arises when you start out needing a package which is a 32 bit package. That package may break if it doesn't have other 32 bit packages to depend on. Yes, of course these things really ought to be interchangeable, but let's remember that this is open source software. There's a real, meaningful trade-off between hard encapsulation and rate of progress. Allowing for greater flexibility encourages a much richer growth pattern. And you aren't paying for this stuff, so cope and install both sets of packages....
Some packages to be aware of specific to 64 bit systems
Using Virtualization to create a server testbed[edit] Virtualization is an important tool for web 2.0 and science 2.0 research labs. Through virtualization, you can test the effects of your changes on your end users in a rich approximation of their environment in an automated fashion, without destabilizing them. Much like Selenium, but at the next level down into the system. A variety of commercial products for specific uses now have virtualization built in to them. For lab uses, though, you generally need to use one of the major virtualization platforms.VMWareVMWare is one of the oldest of the virtualization platforms. It runs on pretty much any OS, and a student version is available for free (yay, VMWare!)XenXen started out as the fedora project's virtualization framework, but has since graduated first to a wide variety of Linux distributions, and then to other Unix distributions. Xen uses modified kernels, so is much less resource intensive than VMWare. Xen's documentation is, shall we say, in flux, as Xen moves from being tightly linked with fedora/Red Hat to being a more widely used system. We offer the following set of resources to help out in the transition... back to topManaging Perl[edit] Any Unix distribution will come with perl installed. Hilary has a bunch of perl info she needs to move over here, where it will be much easier to play with and reorg, but until then, here's the critical info...Bootstrapping Perl[edit] We use yum to bootstrap a perl installation on a fedora machine, as follows (these are just brief notes, you are welcome to do this on your own fedora machine, however if doing this for the first time on a lab machine and unfamiliar with the tools, ask for help.)
Perl's package managers: cpan and cpanp[edit] Perl has its own package managers (see BootstrappingPerl), which should be used in preference to rpm or yum. There are two interfaces (modules) in use right now - a lower layer of abstraction, the CPAN module (MCPAN) and a higher layer of abstraction, the CPANPLUS module (MCPANPLUS) to which the community is migrating, slowly. It's part of a larger migration to 'pure perl' based on a new module abstraction layer (Module::Build). In any case, you should use cpanp for most things, however, for some things you will still need to fall back to perl -MCPAN -e shell. Some hints about using cpanp.
Email services, including Listservs[edit] We don't provide email accounts (and I recommend against doing so, but pocket lab servers still need to run a variety of mail services to meet the needs of other applications. That will require interfacing with your university (or other provider) mailhosts, as well as thinking through and configuring your own services. As we need to maintain our configuration, we'll add more detail to these notes.Mailservers (postfix)[edit] postfix is our prefered mailserver (not sendmail - yuck!). When you install postfix, make sure you not only stop sendmail, but actually remove it from the system and add it to the yum excludes list (YumDotConf). Why use postfix rather than sendmail? The reasons are legion - it's just so much easier, cleaner, safer, more stable. But why trust me? Read for yourself or best of all, try it and see. [back to top]Listservs (MailMan)[edit] We use Mailman for our email lists. Mailman is written in Python, and is easy to use. It has built-in web-based archiving via pipermail archives, which are pretty ugly, but has hooks for use with MHonArc. MHonArc is quite nice. [back to top]Managing Apache[edit] Why Apache rather than other webservers? There are lots of reasons, but I suppose, in the end, it has to do with working on the platform that w3c works on. The same reason that I prefer to use Firefox as my default browser. -- HilaryHolz - 23 May 2008Designing an Apache install - Danger, Will Robinson![edit] The key to a happy and healthy Apache installation is a good design - that's a hard learned lesson. What's more, for pocket labs, that design will change steadily over time, as Apache continues to develop. What we throw out here is what works quite well for us, but we're very open to collaboration. Folks, this is a really key area for us right now due to the current thrust towards virtual labs. Now, virtual labs are really useful, but our lab has learned from experience is that anyone working seriously in our areas of Digital Arts and Sciences is going to run into trouble quite quickly trying to use virtual labs for our work. Our work is simply too close to the OS for VMWare to support. Given that we're not an OS research group, that's rather revealing. The Digital Arts and Sciences folks need to work as a community to collaborate with IT and university administration to help them sort out what sorts of uses virtual labs are good for and what sorts of uses they aren't - not fight the whole trend, or work in isolation, duplicating effort and wasting time that could be put towards our research and teaching Some major recommendations:
Configuring Apache[edit] Config file are generally in /etc/httpd/conf and /etc/httpd/conf.d, however some 3rd party modules have additional configuration files (e.g., modsecurity has a whole set it adds to /etc/modsecurity.d) Our design is agile, i.e., the only configuration information that should go in the main configuration file, /etc/httpd/conf/httpd.conf, is that which applies to the base configuration of apache. Configuration information for dynamically loaded modules ('dso's) should go in separate configuration files in /etc/httpd/conf.d. The main configuration file automatically loads all the rest of the configuration files. Make sure that all configuration information needed for a particular module or application is self-contained in a config file in /etc/httpd/conf.d so that only that config file need be changed if/when the module changes.
PHP[edit] No PHP. Ever. Turn it off. Making and keeping php secure takes more resources than we have. You must get explicit permission from Hilary to turn it on, even for a microsecond, and it will be really tough to talk her into it, as she has better things to do with her time than deal with the resulting continual barrage of attacks. back to topmod_ssl[edit] Comes as part of the source install, although you do have to enable it explicitly. We use openssl as our ssl package. (Get info about how to get certificates signed...) back to topmod_security[edit] Install and use mod_security (keep up to date). Mod_security is a third-party module (not part of the apache project, so you get it from the modsecurity homepage. You'll need to sign up for an account at the breach security network to actually download the source.installing mod_security on fedora
mod_evasive[edit] Jonathan Zdziarski's mod_evasive module is also a fundamental protection module to install on your Apache server. Download it from his site and follow the ridiculously clear and easy directions in the README file Thanks, fella!
back to top
mod_perl[edit] You'll also be best off installing mod_perl from source (and it is easier than trying to use a package install of mod_perl.) The source comes from the mod_perl homepage If you use the configure command listed above, you can configure mod_perl using 'perl Makefile.PL MP_APXS=/usr/sbin/apxs' and then follow the standard install process. If, for some reason, you decide you need more complex configuration options, you should store those options in an options file and add the argument MP_OPTIONS_FILE to the configure step. The first time one of us finds a need to do this, think through a good name and location for a file and come back here and document it. back to topgdbm[edit] As of April 1st, 2008, the rpm install of gdbm had a bug that affected mod_perl, so you'll need to install gdbm from source. You can download it from any GNU mirror, you want version 1.8.3 (which dates from 2002!) back to top . . .ManagingTWiki[edit] 05 Aug 2008: Feel free to dive right in, but you may find this quick foreword very helpful in using these materials. Ahat (the lab) has been maintaining a collaborative set of notes since its inception, in 2001. Initially these were jointly owned and authored webpages, to which we added wiki pages on a kwiki server. Kwiki did not work out, however, we were hooked on wikiservers. As we continued to add collaboration tools to our toolkit (such as Subversion), we explored a number of alternative wikiservers, including several exploratory shakedown deployments, finally settling on twiki. We deployed our initial production version twiki server in May 2008. As such, the information in ManagingTWiki represents the oldest set of topics since our migration to TWiki. Moving to TWiki resulted in an explosion of collaborative activity in the lab, which is wonderful - the truest affirmation we could have had of our decision, as well as pretty great endorsement of TWiki
We've brought on some new interns, Nate and Charles, and are now getting our feet back under ourselves. Phew! Currently, we're reviewing all the materials on the Ahatwiki. We're also upgrading to 4.2.1, so I'm working on bringing these topics up to date, but be aware that both the style and content of some of the ManagingTWiki stuff may lag behind our current information design best practices.
TWiki Configuration Files[edit] TWiki has a variety of configuration files, more than just those that the TWiki folks themselves consider to be configuration files. We list and discuss those files here. As we go along, we'll try to keep our notes up to date. The main benefit of this discussion, though, is to make you aware of where the possible pitfalls are. When upgrading and/or writing TWiki extensions or working on the TWiki core, make sure that you always research the TWiki developer documentation thoroughly for any new information regarding these configuration files before proceeding. Do the rest of us a favor, too, ok? Take a few moments and do a core dump here with your notes and info. Even the raw stuff is great if you don't have time to be coherent, It'll help the next developer immensely. Just mark what level of editing you've had a chance to do when you add your info to the pile...twiki configuration files (as of 05 Aug 2008)
apache twiki configuration files (as of 05 Aug 2008)
About TWiki Skins[edit]twiki skins
installing skins
modifying / customizing skins
TWiki Admin Task Cheat-Sheet[edit] A cheat-sheet for annoying TWiki admin tasks
SQL Database drivers Show...  Hide PostgreSQL:We have a strong preference for PostgreSQL, both for the driver itself and for the community around it. Due to current issues with yum, it's best to install it from source (easy, too.) That way, you get pg_config, for example.MySQL:Has gotten pretty commercial, but also needs to be installed from source.Webmin ConfigShow... Hide
|
Reference Desk
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
r64 - 26 Jun 2009 - 15:36:59 - HilaryHolz
Guests are welcome to view our materials. To subscribe, edit, view raw markup, etc., you'll need to register for an account. Accounts are free (and will always be free) - your involvement helps us directly and indirectly (by demonstrating that our work matters to our funders...) StartingPoints has more info.
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 Copyright 1999-2009 by the contributing authors. All material on this collaboration platform is the property of the contributing authors. Ideas, requests, problems regarding Ahatwiki? Send feedback Syndicate this site RSSATOM |