Index: branches/fc20-dev/COPYRIGHT =================================================================== --- branches/fc20-dev/COPYRIGHT (revision 2523) +++ branches/fc20-dev/COPYRIGHT (revision 2523) @@ -0,0 +1,358 @@ +scripts.mit.edu repository +Copyright (C) 2006 Jeff Arnold and Joe Presbrey, + unless noted otherwise + +These programs are free software; you can redistribute them and/or +modify them under the terms of the GNU General Public License +as published by the Free Software Foundation; either version 2 +of the License, or (at your option) any later version. + +These programs are distributed in the hope that they will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +You should have received a copy of the GNU General Public License +along with this program; if not, write to the Free Software +Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA + + GNU GENERAL PUBLIC LICENSE + Version 2, June 1991 + + Copyright (C) 1989, 1991 Free Software Foundation, Inc. + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +License is intended to guarantee your freedom to share and change free +software--to make sure the software is free for all its users. This +General Public License applies to most of the Free Software +Foundation's software and to any other program whose authors commit to +using it. (Some other Free Software Foundation software is covered by +the GNU Library General Public License instead.) You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. + + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if you +distribute copies of the software, or if you modify it. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must give the recipients all the rights that +you have. You must make sure that they, too, receive or can get the +source code. And you must show them these terms so they know their +rights. + + We protect your rights with two steps: (1) copyright the software, and +(2) offer you this license which gives you legal permission to copy, +distribute and/or modify the software. + + Also, for each author's protection and ours, we want to make certain +that everyone understands that there is no warranty for this free +software. If the software is modified by someone else and passed on, we +want its recipients to know that what they have is not the original, so +that any problems introduced by others will not reflect on the original +authors' reputations. + + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that redistributors of a free +program will individually obtain patent licenses, in effect making the +program proprietary. To prevent this, we have made it clear that any +patent must be licensed for everyone's free use or not licensed at all. + + The precise terms and conditions for copying, distribution and +modification follow. + + GNU GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License applies to any program or other work which contains +a notice placed by the copyright holder saying it may be distributed +under the terms of this General Public License. The "Program", below, +refers to any such program or work, and a "work based on the Program" +means either the Program or any derivative work under copyright law: +that is to say, a work containing the Program or a portion of it, +either verbatim or with modifications and/or translated into another +language. (Hereinafter, translation is included without limitation in +the term "modification".) Each licensee is addressed as "you". + +Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running the Program is not restricted, and the output from the Program +is covered only if its contents constitute a work based on the +Program (independent of having been made by running the Program). +Whether that is true depends on what the Program does. + + 1. You may copy and distribute verbatim copies of the Program's +source code as you receive it, in any medium, provided that you +conspicuously and appropriately publish on each copy an appropriate +copyright notice and disclaimer of warranty; keep intact all the +notices that refer to this License and to the absence of any warranty; +and give any other recipients of the Program a copy of this License +along with the Program. + +You may charge a fee for the physical act of transferring a copy, and +you may at your option offer warranty protection in exchange for a fee. + + 2. You may modify your copy or copies of the Program or any portion +of it, thus forming a work based on the Program, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) You must cause the modified files to carry prominent notices + stating that you changed the files and the date of any change. + + b) You must cause any work that you distribute or publish, that in + whole or in part contains or is derived from the Program or any + part thereof, to be licensed as a whole at no charge to all third + parties under the terms of this License. + + c) If the modified program normally reads commands interactively + when run, you must cause it, when started running for such + interactive use in the most ordinary way, to print or display an + announcement including an appropriate copyright notice and a + notice that there is no warranty (or else, saying that you provide + a warranty) and that users may redistribute the program under + these conditions, and telling the user how to view a copy of this + License. (Exception: if the Program itself is interactive but + does not normally print such an announcement, your work based on + the Program is not required to print an announcement.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Program, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Program, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Program. + +In addition, mere aggregation of another work not based on the Program +with the Program (or with a work based on the Program) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may copy and distribute the Program (or a work based on it, +under Section 2) in object code or executable form under the terms of +Sections 1 and 2 above provided that you also do one of the following: + + a) Accompany it with the complete corresponding machine-readable + source code, which must be distributed under the terms of Sections + 1 and 2 above on a medium customarily used for software interchange; or, + + b) Accompany it with a written offer, valid for at least three + years, to give any third party, for a charge no more than your + cost of physically performing source distribution, a complete + machine-readable copy of the corresponding source code, to be + distributed under the terms of Sections 1 and 2 above on a medium + customarily used for software interchange; or, + + c) Accompany it with the information you received as to the offer + to distribute corresponding source code. (This alternative is + allowed only for noncommercial distribution and only if you + received the program in object code or executable form with such + an offer, in accord with Subsection b above.) + +The source code for a work means the preferred form of the work for +making modifications to it. For an executable work, complete source +code means all the source code for all modules it contains, plus any +associated interface definition files, plus the scripts used to +control compilation and installation of the executable. However, as a +special exception, the source code distributed need not include +anything that is normally distributed (in either source or binary +form) with the major components (compiler, kernel, and so on) of the +operating system on which the executable runs, unless that component +itself accompanies the executable. + +If distribution of executable or object code is made by offering +access to copy from a designated place, then offering equivalent +access to copy the source code from the same place counts as +distribution of the source code, even though third parties are not +compelled to copy the source along with the object code. + + 4. You may not copy, modify, sublicense, or distribute the Program +except as expressly provided under this License. Any attempt +otherwise to copy, modify, sublicense or distribute the Program is +void, and will automatically terminate your rights under this License. +However, parties who have received copies, or rights, from you under +this License will not have their licenses terminated so long as such +parties remain in full compliance. + + 5. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Program or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Program (or any work based on the +Program), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Program or works based on it. + + 6. Each time you redistribute the Program (or any work based on the +Program), the recipient automatically receives a license from the +original licensor to copy, distribute or modify the Program subject to +these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. + + 7. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Program at all. For example, if a patent +license would not permit royalty-free redistribution of the Program by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Program. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system, which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 8. If the distribution and/or use of the Program is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Program under this License +may add an explicit geographical distribution limitation excluding +those countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 9. The Free Software Foundation may publish revised and/or new versions +of the General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + +Each version is given a distinguishing version number. If the Program +specifies a version number of this License which applies to it and "any +later version", you have the option of following the terms and conditions +either of that version or of any later version published by the Free +Software Foundation. If the Program does not specify a version number of +this License, you may choose any version ever published by the Free Software +Foundation. + + 10. If you wish to incorporate parts of the Program into other free +programs whose distribution conditions are different, write to the author +to ask for permission. For software which is copyrighted by the Free +Software Foundation, write to the Free Software Foundation; we sometimes +make exceptions for this. Our decision will be guided by the two goals +of preserving the free status of all derivatives of our free software and +of promoting the sharing and reuse of software generally. + + NO WARRANTY + + 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY +FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN +OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES +PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED +OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS +TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE +PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, +REPAIR OR CORRECTION. + + 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR +REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, +INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING +OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED +TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY +YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER +PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA + + +Also add information on how to contact you by electronic and paper mail. + +If the program is interactive, make it output a short notice like this +when it starts in an interactive mode: + + Gnomovision version 69, Copyright (C) year name of author + Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, the commands you use may +be called something other than `show w' and `show c'; they could even be +mouse-clicks or menu items--whatever suits your program. + +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the program, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the program + `Gnomovision' (which makes passes at compilers) written by James Hacker. + + , 1 April 1989 + Ty Coon, President of Vice + +This General Public License does not permit incorporating your program into +proprietary programs. If your program is a subroutine library, you may +consider it more useful to permit linking proprietary applications with the +library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. Index: branches/fc20-dev/README =================================================================== --- branches/fc20-dev/README (revision 2523) +++ branches/fc20-dev/README (revision 2523) @@ -0,0 +1,11 @@ +host: + files needed to set up a scripts.mit.edu hypervisor (aka VM host) + +locker: + files associated with the scripts Athena locker + +lvs: + files needed to set up a scripts.mit.edu director (aka load balancer) + +server: + files needed to run a scripts.mit.edu server (aka realserver) Index: branches/fc20-dev/host/credit-card/host.py =================================================================== --- branches/fc20-dev/host/credit-card/host.py (revision 2523) +++ branches/fc20-dev/host/credit-card/host.py (revision 2523) @@ -0,0 +1,239 @@ +import os +import optparse +import socket +import tempfile +import shutil +import errno +import csv + +import shell + +HOST = socket.gethostname() + +PROD_GUESTS = frozenset([ + 'bees-knees', + 'cats-whiskers', + 'busy-beaver', + 'pancake-bunny', + 'whole-enchilada', + 'real-mccoy', + 'old-faithful', + 'better-mousetrap', + 'shining-armor', + 'golden-egg', + 'miracle-cure', + 'lucky-star', + ]) +WIZARD_GUESTS = frozenset([ + 'not-backward', + ]) + +COMMON_CREDS = {} + +# Format here assumes that we always chmod $USER:$USER, +# but note the latter refers to group... +# +# Important: no leading slashes! +COMMON_CREDS['all'] = [ + ('root', 0o600, 'root/.bashrc'), + ('root', 0o600, 'root/.screenrc'), + ('root', 0o600, 'root/.ssh/authorized_keys'), + ('root', 0o600, 'root/.ssh/authorized_keys2'), + ('root', 0o600, 'root/.vimrc'), + ('root', 0o600, 'root/.k5login'), + ] + +COMMON_CREDS['prod'] = [ + ('root', 0o600, 'root/.ldapvirc'), + ('root', 0o600, 'etc/ssh/ssh_host_dsa_key'), + ('root', 0o600, 'etc/ssh/ssh_host_key'), + ('root', 0o600, 'etc/ssh/ssh_host_rsa_key'), + ('root', 0o600, 'etc/pki/tls/private/scripts-1024.key'), + ('root', 0o600, 'etc/pki/tls/private/scripts.key'), + ('root', 0o600, 'etc/whoisd-password'), + ('afsagent', 0o600, 'etc/daemon.keytab'), + + ('root', 0o644, 'etc/ssh/ssh_host_dsa_key.pub'), + ('root', 0o644, 'etc/ssh/ssh_host_key.pub'), + ('root', 0o644, 'etc/ssh/ssh_host_rsa_key.pub'), + + ('sql', 0o600, 'etc/sql-mit-edu.cfg.php'), # technically doesn't have to be secret anymore + ('sql', 0o600, 'etc/sql-password'), + ('signup', 0o600, 'etc/signup-ldap-pw'), + ('logview', 0o600, 'home/logview/.k5login'), # XXX user must be created in Kickstart + ] + +# note that these are duplicates with 'prod', but the difference +# is that the files DIFFER between wizard and prod +COMMON_CREDS['wizard'] = [ + ('root', 0o600, 'etc/ssh/ssh_host_dsa_key'), + ('root', 0o600, 'etc/ssh/ssh_host_key'), + ('root', 0o600, 'etc/ssh/ssh_host_rsa_key'), + ('afsagent', 0o600, 'etc/daemon.keytab'), + + ('root', 0o644, 'etc/ssh/ssh_host_dsa_key.pub'), + ('root', 0o644, 'etc/ssh/ssh_host_key.pub'), + ('root', 0o644, 'etc/ssh/ssh_host_rsa_key.pub'), + ] + +MACHINE_CREDS = {} + +MACHINE_CREDS['all'] = [ + # XXX NEED TO CHECK THAT THE CONTENTS ARE SENSIBLE + ('root', 0o600, 'etc/krb5.keytab'), + ] + +MACHINE_CREDS['prod'] = [ + ('fedora-ds', 0o600, 'etc/dirsrv/keytab'), + ] + +MACHINE_CREDS['wizard'] = [] + +# Works for passwd and group, but be careful! They're different things! +def lookup(filename): + # Super-safe to assume and volume IDs (expensive to check) + r = { + 'root': 0, + 'sql': 537704221, + } + with open(filename, 'rb') as f: + reader = csv.reader(f, delimiter=':', quoting=csv.QUOTE_NONE) + for row in reader: + r[row[0]] = int(row[2]) + return r + +def drop_caches(): + with open("/proc/sys/vm/drop_caches", 'w') as f: + f.write("1") + +def mkdir_p(path): # it's like mkdir -p + try: + os.makedirs(path) + except OSError as e: + if e.errno == errno.EEXIST: + pass + else: raise + +# XXX This code is kind of dangerous, because we are directly using the +# kernel modules to manipulate possibly untrusted disk images. This +# means that if an attacker can corrupt the disk, and exploit a problem +# in the kernel vfs driver, he can escalate a guest root exploit +# to a host root exploit. Ultimately we should use libguestfs +# which makes this attack harder to pull off, but at the time of writing +# squeeze didn't package libguestfs. +# +# We try to minimize attack surface by explicitly specifying the +# expected filesystem type. +class WithMount(object): + """Context for running code with an extra mountpoint.""" + guest = None + types = None # comma separated, like the mount argument -t + mount = None + dev = None + def __init__(self, guest, types): + self.guest = guest + self.types = types + def __enter__(self): + drop_caches() + self.dev = "/dev/%s/%s-root" % (HOST, self.guest) + + mapper_name = shell.eval("kpartx", "-l", self.dev).split()[0] + shell.call("kpartx", "-a", self.dev) + mapper = "/dev/mapper/%s" % mapper_name + + # this is why bracketing functions and hanging lambdas are a good idea + try: + self.mount = tempfile.mkdtemp("-%s" % self.guest, 'vm-', '/mnt') # no trailing slash + try: + shell.call("mount", "--types", self.types, mapper, self.mount) + except: + os.rmdir(self.mount) + raise + except: + shell.call("kpartx", "-d", self.dev) + raise + + return self.mount + def __exit__(self, _type, _value, _traceback): + shell.call("umount", self.mount) + os.rmdir(self.mount) + shell.call("kpartx", "-d", self.dev) + drop_caches() + +def main(): + usage = """usage: %prog [push|pull] [common|machine] GUEST""" + + parser = optparse.OptionParser(usage) + # ext3 will probably supported for a while yet and a pretty + # reasonable thing to always try + parser.add_option('-t', '--types', dest="types", default="ext4,ext3", + help="filesystem type(s)") # same arg as 'mount' + parser.add_option('--creds-dir', dest="creds_dir", default="/root/creds", + help="directory to store/fetch credentials in") + options, args = parser.parse_args() + + if not os.path.isdir(options.creds_dir): + raise Exception("%s does not exist" % options.creds_dir) + # XXX check owned by root and appropriately chmodded + + os.umask(0o077) # overly restrictive + + if len(args) != 3: + parser.print_help() + raise Exception("Wrong number of arguments") + + command = args[0] + files = args[1] + guest = args[2] + + if guest in PROD_GUESTS: + mode = 'prod' + elif guest in WIZARD_GUESTS: + mode = 'wizard' + else: + raise Exception("Unrecognized guest %s" % guest) + + with WithMount(guest, options.types) as tmp_mount: + uid_lookup = lookup("%s/etc/passwd" % tmp_mount) + gid_lookup = lookup("%s/etc/group" % tmp_mount) + def push_files(files, type): + for (usergroup, perms, f) in files: + dest = "%s/%s" % (tmp_mount, f) + mkdir_p(os.path.dirname(dest)) # useful for .ssh + # assuming OK to overwrite + # XXX we could compare the files before doing anything... + shutil.copyfile("%s/%s/%s" % (options.creds_dir, type, f), dest) + try: + os.chown(dest, uid_lookup[usergroup], gid_lookup[usergroup]) + os.chmod(dest, perms) + except: + # never ever leave un-chowned files lying around + os.unlink(dest) + raise + def pull_files(files, type): + for (_, _, f) in files: + dest = "%s/%s/%s" % (options.creds_dir, type, f) + mkdir_p(os.path.dirname(dest)) + # error if doesn't exist + shutil.copyfile("%s/%s" % (tmp_mount, f), dest) + + # XXX ideally we should check these *before* we mount, but Python + # makes that pretty annoying to do + if command == "push": + run = push_files + elif command == "pull": + run = pull_files + else: + raise Exception("Unknown command %s, valid values are 'push' and 'pull'" % command) + + if files == 'common': + run(COMMON_CREDS['all'], 'all') + run(COMMON_CREDS[mode], mode) + elif files == 'machine': + run(MACHINE_CREDS['all'], 'machine/%s' % guest) + run(MACHINE_CREDS[mode], 'machine/%s' % guest) + else: + raise Exception("Unknown file set %s, valid values are 'common' and 'machine'" % files) + +if __name__ == "__main__": + main() Index: branches/fc20-dev/host/credit-card/shell.py =================================================================== --- branches/fc20-dev/host/credit-card/shell.py (revision 2523) +++ branches/fc20-dev/host/credit-card/shell.py (revision 2523) @@ -0,0 +1,301 @@ +""" +Wrappers around subprocess functionality that simulate an actual shell. +""" + +import subprocess +import logging +import sys +import os +import errno + +class Shell(object): + """ + An advanced shell that performs logging. If ``dry`` is ``True``, + no commands are actually run. + """ + def __init__(self, dry = False): + self.dry = dry + self.cwd = None + def call(self, *args, **kwargs): + """ + Performs a system call. The actual executable and options should + be passed as arguments to this function. Several keyword arguments + are also supported: + + :param input: input to feed the subprocess on standard input. + :param interactive: whether or not directly hook up all pipes + to the controlling terminal, to allow interaction with subprocess. + :param strip: if ``True``, instead of returning a tuple, + return the string stdout output of the command with trailing newlines + removed. This emulates the behavior of backticks and ``$()`` in Bash. + Prefer to use :meth:`eval` instead (you should only need to explicitly + specify this if you are using another wrapper around this function). + :param log: if True, we log the call as INFO, if False, we log the call + as DEBUG, otherwise, we detect based on ``strip``. + :param stdout: + :param stderr: + :param stdin: a file-type object that will be written to or read from as a pipe. + :returns: a tuple of strings ``(stdout, stderr)``, or a string ``stdout`` + if ``strip`` is specified. + + >>> sh = Shell() + >>> sh.call("echo", "Foobar") + ('Foobar\\n', '') + >>> sh.call("cat", input='Foobar') + ('Foobar', '') + """ + self._wait() + kwargs.setdefault("interactive", False) + kwargs.setdefault("strip", False) + kwargs.setdefault("python", None) + kwargs.setdefault("log", None) + kwargs.setdefault("stdout", subprocess.PIPE) + kwargs.setdefault("stdin", subprocess.PIPE) + kwargs.setdefault("stderr", subprocess.PIPE) + msg = "Running `" + ' '.join(args) + "`" + if kwargs["strip"] and not kwargs["log"] is True or kwargs["log"] is False: + logging.debug(msg) + else: + logging.info(msg) + if self.dry: + if kwargs["strip"]: + return '' + return None, None + kwargs.setdefault("input", None) + if kwargs["interactive"]: + stdout=sys.stdout + stdin=sys.stdin + stderr=sys.stderr + else: + stdout=kwargs["stdout"] + stdin=kwargs["stdin"] + stderr=kwargs["stderr"] + # XXX: There is a possible problem here where we can fill up + # the kernel buffer if we have 64KB of data. This shouldn't + # be a problem, and the fix for such case would be to write to + # temporary files instead of a pipe. + # Another possible way of fixing this is converting from a + # waitpid() pump to a select() pump, creating a pipe to + # ourself, and then setting up a + # SIGCHILD handler to write a single byte to the pipe to get + # us out of select() when a subprocess exits. + proc = subprocess.Popen(args, stdout=stdout, stderr=stderr, stdin=stdin, cwd=self.cwd, ) + if self._async(proc, args, **kwargs): + return proc + stdout, stderr = proc.communicate(kwargs["input"]) + # can occur if we were doing interactive communication; i.e. + # we didn't pass in PIPE. + if stdout is None: + stdout = "" + if stderr is None: + stderr = "" + if not kwargs["interactive"]: + if kwargs["strip"]: + self._log(None, stderr) + else: + self._log(stdout, stderr) + if proc.returncode: + raise CallError(proc.returncode, args, stdout, stderr) + if kwargs["strip"]: + return str(stdout).rstrip("\n") + return (stdout, stderr) + def _log(self, stdout, stderr): + """Logs the standard output and standard input from a command.""" + if stdout: + logging.debug("STDOUT:\n" + stdout) + if stderr: + logging.debug("STDERR:\n" + stderr) + def _wait(self): + pass + def _async(self, *args, **kwargs): + return False + def callAsUser(self, *args, **kwargs): + """ + Performs a system call as a different user. This is only possible + if you are running as root. Keyword arguments + are the same as :meth:`call` with the following additions: + + :param user: name of the user to run command as. + :param uid: uid of the user to run command as. + + .. note:: + + The resulting system call internally uses :command:`sudo`, + and as such environment variables will get scrubbed. We + manually preserve :envvar:`SSH_GSSAPI_NAME`. + """ + user = kwargs.pop("user", None) + uid = kwargs.pop("uid", None) + if not user and not uid: return self.call(*args, **kwargs) + if os.getenv("SSH_GSSAPI_NAME"): + # This might be generalized as "preserve some environment" + args = list(args) + args.insert(0, "SSH_GSSAPI_NAME=" + os.getenv("SSH_GSSAPI_NAME")) + if uid: return self.call("sudo", "-u", "#" + str(uid), *args, **kwargs) + if user: return self.call("sudo", "-u", user, *args, **kwargs) + def safeCall(self, *args, **kwargs): + """ + Checks if the owner of the current working directory is the same + as the current user, and if it isn't, attempts to sudo to be + that user. The intended use case is for calling Git commands + when running as root, but this method should be used when + interfacing with any moderately complex program that depends + on working directory context. Keyword arguments are the + same as :meth:`call`. + """ + if os.getuid(): + return self.call(*args, **kwargs) + uid = os.stat(os.getcwd()).st_uid + # consider also checking ruid? + if uid != os.geteuid(): + kwargs['uid'] = uid + return self.callAsUser(*args, **kwargs) + else: + return self.call(*args, **kwargs) + def eval(self, *args, **kwargs): + """ + Evaluates a command and returns its output, with trailing newlines + stripped (like backticks in Bash). This is a convenience method for + calling :meth:`call` with ``strip``. + + >>> sh = Shell() + >>> sh.eval("echo", "Foobar") + 'Foobar' + """ + kwargs["strip"] = True + return self.call(*args, **kwargs) + def setcwd(self, cwd): + """ + Sets the directory processes are executed in. This sets a value + to be passed as the ``cwd`` argument to ``subprocess.Popen``. + """ + self.cwd = cwd + +class ParallelShell(Shell): + """ + Modifies the semantics of :class:`Shell` so that + commands are queued here, and executed in parallel using waitpid + with ``max`` subprocesses, and result in callback execution + when they finish. + + .. method:: call(*args, **kwargs) + + Enqueues a system call for parallel processing. If there are + no openings in the queue, this will block. Keyword arguments + are the same as :meth:`Shell.call` with the following additions: + + :param on_success: Callback function for success (zero exit status). + The callback function should accept two arguments, + ``stdout`` and ``stderr``. + :param on_error: Callback function for failure (nonzero exit status). + The callback function should accept one argument, the + exception that would have been thrown by the synchronous + version. + :return: The :class:`subprocess.Proc` object that was opened. + + .. method:: callAsUser(*args, **kwargs) + + Enqueues a system call under a different user for parallel + processing. Keyword arguments are the same as + :meth:`Shell.callAsUser` with the additions of keyword + arguments from :meth:`call`. + + .. method:: safeCall(*args, **kwargs) + + Enqueues a "safe" call for parallel processing. Keyword + arguments are the same as :meth:`Shell.safeCall` with the + additions of keyword arguments from :meth:`call`. + + .. method:: eval(*args, **kwargs) + + No difference from :meth:`call`. Consider having a + non-parallel shell if the program you are shelling out + to is fast. + + """ + def __init__(self, dry = False, max = 10): + super(ParallelShell, self).__init__(dry=dry) + self.running = {} + self.max = max # maximum of commands to run in parallel + @staticmethod + def make(no_parallelize, max): + """Convenience method oriented towards command modules.""" + if no_parallelize: + return DummyParallelShell() + else: + return ParallelShell(max=max) + def _async(self, proc, args, python, on_success, on_error, **kwargs): + """ + Gets handed a :class:`subprocess.Proc` object from our deferred + execution. See :meth:`Shell.call` source code for details. + """ + self.running[proc.pid] = (proc, args, python, on_success, on_error) + return True # so that the parent function returns + def _wait(self): + """ + Blocking call that waits for an open subprocess slot. This is + automatically called by :meth:`Shell.call`. + """ + # XXX: This API sucks; the actual call/callAsUser call should + # probably block automatically (unless I have a good reason not to) + # bail out immediately on initial ramp up + if len(self.running) < self.max: return + # now, wait for open pids. + try: + self.reap(*os.waitpid(-1, 0)) + except OSError as e: + if e.errno == errno.ECHILD: return + raise + def join(self): + """Waits for all of our subprocesses to terminate.""" + try: + while True: + self.reap(*os.waitpid(-1, 0)) + except OSError as e: + if e.errno == errno.ECHILD: return + raise + def reap(self, pid, status): + """Reaps a process.""" + # ooh, zombie process. reap it + proc, args, python, on_success, on_error = self.running.pop(pid) + # XXX: this is slightly dangerous; should actually use + # temporary files + stdout = proc.stdout.read() + stderr = proc.stderr.read() + self._log(stdout, stderr) + if status: + on_error(CallError(proc.returncode, args, stdout, stderr)) + return + on_success(stdout, stderr) + +# Setup a convenience global instance +shell = Shell() +call = shell.call +callAsUser = shell.callAsUser +safeCall = shell.safeCall +eval = shell.eval + +class DummyParallelShell(ParallelShell): + """Same API as :class:`ParallelShell`, but doesn't actually + parallelize (i.e. all calls to :meth:`wait` block.)""" + def __init__(self, dry = False): + super(DummyParallelShell, self).__init__(dry=dry, max=1) + +class CallError: + """Indicates that a subprocess call returned a nonzero exit status.""" + #: The exit code of the failed subprocess. + code = None + #: List of the program and arguments that failed. + args = None + #: The stdout of the program. + stdout = None + #: The stderr of the program. + stderr = None + def __init__(self, code, args, stdout, stderr): + self.code = code + self.args = args + self.stdout = stdout + self.stderr = stderr + def __str__(self): + compact = self.stderr.rstrip().split("\n")[-1] + return "%s (exited with %d)\n%s" % (compact, self.code, self.stderr) Index: branches/fc20-dev/host/debian/scripts-syslog-ng-config/d_zroot.pl =================================================================== --- branches/fc20-dev/host/debian/scripts-syslog-ng-config/d_zroot.pl (revision 2523) +++ branches/fc20-dev/host/debian/scripts-syslog-ng-config/d_zroot.pl (revision 2523) @@ -0,0 +1,149 @@ +#!/usr/bin/perl + +use strict; +use warnings; +use Sys::Hostname; +use Time::HiRes qw(ualarm); +use File::Temp; + +our $ZCLASS = "scripts-auto"; +our @USERS = qw/root logview/; +my $k5login; +open $k5login, '/root/.k5login'; +our @RECIPIENTS = map {chomp; m|([^/@]*)| && $1} <$k5login>; +close $k5login; + +our %USERS; +@USERS{@USERS} = undef; + +sub zwrite($;$$\@) { + my ($message, $class, $instance, $recipref) = @_; + my @recipients = (); + if (defined($recipref)) { + if (@$recipref) { + @recipients = @$recipref; + } else { + $message = '@b(Empty recipient list specified, message redacted)'; + $class = $ZCLASS; + } + } + $class ||= $ZCLASS; + $instance ||= 'root.'.hostname; + open(ZWRITE, "|-", qw|/usr/bin/zwrite -d -n -O log -c|, $class, '-i', $instance, '-s', hostname, @recipients) or die "Couldn't open zwrite"; + print ZWRITE $message; + close(ZWRITE); +} + +unless (@RECIPIENTS) { + # Also give a warning at startup + zwrite('@b(No .k5login found, sensitive logs will not be zephyred)', $ZCLASS); +} + +my %toclass; + +my %sshkeys; + +sub buildKeyMap($) { + my ($file) = @_; + open (KEYS, $file) or (warn "Couldn't open $file: $!\n" and return); + while () { + chomp; + my ($fingerprint, $comment) = parseKey($_); + $sshkeys{$fingerprint} = $comment; + } + close(KEYS); +} + +sub parseKey($) { + my ($key) = @_; + my $tmp = new File::Temp; + print $tmp $key; + close $tmp; + open (KEYGEN, "-|", qw(/usr/bin/ssh-keygen -l -f), $tmp) or die "Couldn't call ssh-keygen: $!"; + my ($line) = ; + close(KEYGEN); + my (undef, $fingerprint, undef) = split(' ', $line, 3); + my (undef, undef, $comment) = split(' ', $key, 3); + #print "$fingerprint $comment"; + return ($fingerprint, $comment); +} + +buildKeyMap("/root/.ssh/authorized_keys"); +buildKeyMap("/root/.ssh/authorized_keys2"); + +my @message; + +while (my $line = <>) { + @message = $line; + eval { + local $SIG{ALRM} = sub { die "alarm\n" }; # NB: \n required + ualarm(500*1000); + while (<>) { push @message, $_; } + }; + chomp @message; + map { s/^(.*?): // } @message; + %toclass = (); + foreach my $message (@message) { + sub sendmsg ($;$) { + my ($message, $class) = @_; + $class ||= $ZCLASS; + $toclass{$class} .= $message."\n"; + } + if ($message =~ m|Accepted (\S+) for (\S+)|) { + sendmsg($message) if exists $USERS{$2} + } elsif ($message =~ m|Authorized to (\S+),|) { + sendmsg($message) if exists $USERS{$1}; + } elsif ($message =~ m|Root (\S+) shell|) { + sendmsg($message); + } elsif ($message =~ m|pam_unix\(([^:]+):session\): session \S+ for user (\S+)|) { + sendmsg($message) if $1 ne "cron" and exists $USERS{$2}; + } elsif ($message =~ m|^Found matching (\w+) key: (\S+)|) { + if ($sshkeys{$2}) { + sendmsg($message." (".$sshkeys{$2}.")"); + } else { + sendmsg($message." (UNKNOWN KEY)"); + } + } elsif ($message =~ m|^Out of memory:|) { + sendmsg($message); + } elsif ($message =~ m|^giving \S+ admin rights|) { + sendmsg($message); + } elsif ($message =~ m|^Connection closed|) { + # Do nothing + } elsif ($message =~ m|^Closing connection to |) { + } elsif ($message =~ m|^Connection from (\S+) port (\S+)|) { + } elsif ($message =~ m|^Invalid user|) { + } elsif ($message =~ m|^input_userauth_request: invalid user|) { + } elsif ($message =~ m|^Received disconnect from|) { + } elsif ($message =~ m|^Postponed keyboard-interactive|) { + } elsif ($message =~ m|^Failed keyboard-interactive/pam|) { + } elsif ($message =~ m|^fatal: Read from socket failed: Connection reset by peer$|) { + } elsif ($message =~ m|^reverse mapping checking getaddrinfo|) { + } elsif ($message =~ m|^pam_succeed_if\(sshd\:auth\)\:|) { + } elsif ($message =~ m|^error: PAM: Authentication failure|) { + } elsif ($message =~ m|^pam_unix\(sshd:auth\): authentication failure|) { + } elsif ($message =~ m|^pam_unix\(sshd:auth\): check pass; user unknown|) { + } elsif ($message =~ m|^Postponed keyboard-interactive for invalid user |) { + } elsif ($message =~ m|^Failed keyboard-interactive/pam for invalid user |) { + } elsif ($message =~ m|^Postponed gssapi-with-mic for |) { + } elsif ($message =~ m|^Address \S+ maps to \S+, but this does not map back to the address|) { + } elsif ($message =~ m|^Nasty PTR record .* is set up for .*, ignoring|) { + } elsif ($message =~ m|^User child is on pid \d+$|) { + } elsif ($message =~ m|^Transferred: sent \d+, received \d+ bytes$|) { + } elsif ($message =~ m|^Setting tty modes failed: Invalid argument$|) { + } elsif ($message =~ m|^ *nrpe .* COMMAND=/etc/nagios/check_ldap_mmr.real$|) { + } elsif ($message =~ m|^ *root : TTY=|) { + } elsif ($message =~ m|^Set /proc/self/oom_adj to |) { + } elsif ($message =~ m|^fatal: mm_request_receive: read: Connection reset by peer$|) { + } else { + sendmsg($message, "scripts-spew"); + } + } + + foreach my $class (keys %toclass) { + if ($class eq $ZCLASS) { + zwrite($toclass{$class}, $class); + } else { + zwrite($toclass{$class}, $class, undef, @RECIPIENTS); + } + } +} Index: branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/changelog =================================================================== --- branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/changelog (revision 2523) +++ branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/changelog (revision 2523) @@ -0,0 +1,37 @@ +scripts-syslog-ng-config (0.6) wheezy; urgency=low + + * Take into account that config-package-deb on dh7 requires .transform files + + -- Alexander Chernyakhovsky Sun, 26 May 2013 22:36:12 -0400 + +scripts-syslog-ng-config (0.5) wheezy; urgency=low + + * Update to dh7 + + -- Alexander Chernyakhovsky Sun, 26 May 2013 21:49:26 -0400 + +scripts-syslog-ng-config (0.4) stable; urgency=low + + * Synchronize filtered logs with Fedora copy (r2095). + * Avoid accidentally sending sensitive logs to a public class (r2096). + + -- Geoffrey Thomas Sat, 17 Dec 2011 03:06:59 -0500 + +scripts-syslog-ng-config (0.3) stable; urgency=low + + * Depend on debathena-zephyr-config + + -- Alexander Chernyakhovsky Sat, 12 Nov 2011 23:27:51 -0500 + +scripts-syslog-ng-config (0.2) stable; urgency=low + + * Move d_zroot into /usr/lib + + -- Quentin Smith Mon, 05 Sep 2011 15:01:04 -0400 + +scripts-syslog-ng-config (0.1) stable; urgency=low + + * Initial release + + -- Alexander Chernyakhovsky Mon, 05 Sep 2011 14:45:27 -0400 + Index: branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/compat =================================================================== --- branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/compat (revision 2523) +++ branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/compat (revision 2523) @@ -0,0 +1,1 @@ +7 Index: branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/control =================================================================== --- branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/control (revision 2523) +++ branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/control (revision 2523) @@ -0,0 +1,14 @@ +Source: scripts-syslog-ng-config +Section: misc +Priority: extra +Maintainer: scripts team +Build-Depends: debhelper (>= 7.0.50~), config-package-dev, syslog-ng, +Standards-Version: 3.9.4 +Homepage: http://scripts.mit.edu/ + +Package: scripts-syslog-ng-config +Architecture: all +Depends: ${misc:Depends}, + syslog-ng, debathena-zephyr-config +Description: Configures syslog-ng for zephyr logging + Configures a machine to log messages sent to syslog-ng to zephyr. Index: branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/copyright =================================================================== --- branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/copyright (revision 2523) +++ branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/copyright (revision 2523) @@ -0,0 +1,22 @@ +This work was packaged for Debian by: + + Alexander Chernyakhovsky on Mon, 05 Sep 2011 14:42:37 -0400 + +It was downloaded from: + + http://scripts.mit.edu + +Copyright: + + Copyright (C) 2011 Alexander Chernyakhovsky + +License: + + GPLv2+ + +The Debian packaging is: + + Copyright (C) 2011 Alexander Chernyakhovsky + +and is licensed under the GPL version 2 +see "/usr/share/common-licenses/GPL-2". Index: branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/rules =================================================================== --- branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/rules (revision 2523) +++ branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/rules (revision 2523) @@ -0,0 +1,3 @@ +#!/usr/bin/make -f +%: + dh $@ --with config-package Index: branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/scripts-syslog-ng-config.install =================================================================== --- branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/scripts-syslog-ng-config.install (revision 2523) +++ branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/scripts-syslog-ng-config.install (revision 2523) @@ -0,0 +1,1 @@ +d_zroot.pl /usr/lib/scripts-syslog-ng-config/ Index: branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/scripts-syslog-ng-config.postinst =================================================================== --- branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/scripts-syslog-ng-config.postinst (revision 2523) +++ branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/scripts-syslog-ng-config.postinst (revision 2523) @@ -0,0 +1,44 @@ +#!/bin/sh +# postinst script for scripts-syslog-ng-config +# +# see: dh_installdeb(1) + +set -e + +# summary of how this script can be called: +# * `configure' +# * `abort-upgrade' +# * `abort-remove' `in-favour' +# +# * `abort-remove' +# * `abort-deconfigure' `in-favour' +# `removing' +# +# for details, see http://www.debian.org/doc/debian-policy/ or +# the debian-policy package + + +case "$1" in + configure) + if hash invoke-rc.d > /dev/null 2>&1; then + invoke-rc.d syslog-ng restart || : + else + /etc/init.d/syslog-ng restart || : + fi + ;; + + abort-upgrade|abort-remove|abort-deconfigure) + ;; + + *) + echo "postinst called with unknown argument \`$1'" >&2 + exit 1 + ;; +esac + +# dh_installdeb will replace this with shell code automatically +# generated by other debhelper scripts. + +#DEBHELPER# + +exit 0 Index: branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/transform_syslog-ng.conf.scripts =================================================================== --- branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/transform_syslog-ng.conf.scripts (revision 2523) +++ branches/fc20-dev/host/debian/scripts-syslog-ng-config/debian/transform_syslog-ng.conf.scripts (revision 2523) @@ -0,0 +1,11 @@ +#!/bin/sh + +cat +cat < Sun, 26 May 2013 22:44:41 -0400 + +scripts-vm-host (0.7) wheezy; urgency=low + + * emacs24-nox is a lie for now, switch back to emacs23-nox + * Switch to dh7 + + -- Alexander Chernyakhovsky Sun, 26 May 2013 21:24:31 -0400 + +scripts-vm-host (0.6) wheezy; urgency=low + + * Update configuration for wheezy + + -- Alexander Chernyakhovsky Sun, 26 May 2013 20:59:04 -0400 + +scripts-vm-host (0.5) stable; urgency=low + + * Add useful dependencies + + -- Quentin Smith Sun, 11 Sep 2011 23:44:32 -0400 + +scripts-vm-host (0.4) stable; urgency=low + + * Depend on scripts-syslog-ng-config + + -- Alexander Chernaykhovsky Mon, 05 Sep 2011 15:19:01 -0400 + +scripts-vm-host (0.3) stable; urgency=low + + * Rebuild for inclusion in Scripts APT repository + + -- Alexander Chernyakhovsky Sun, 04 Sep 2011 20:57:55 -0400 + +scripts-vm-host (0.2) unstable; urgency=low + + * Uhh, actually depend on c-p-d + + -- Quentin Smith Sat, 18 Jun 2011 15:36:44 -0400 + +scripts-vm-host (0.1) unstable; urgency=low + + * Initial release + + -- Quentin Smith Wed, 08 Jun 2011 23:22:31 -0400 Index: branches/fc20-dev/host/debian/scripts-vm-host/debian/compat =================================================================== --- branches/fc20-dev/host/debian/scripts-vm-host/debian/compat (revision 2523) +++ branches/fc20-dev/host/debian/scripts-vm-host/debian/compat (revision 2523) @@ -0,0 +1,1 @@ +7 Index: branches/fc20-dev/host/debian/scripts-vm-host/debian/control =================================================================== --- branches/fc20-dev/host/debian/scripts-vm-host/debian/control (revision 2523) +++ branches/fc20-dev/host/debian/scripts-vm-host/debian/control (revision 2523) @@ -0,0 +1,52 @@ +Source: scripts-vm-host +Section: misc +Priority: extra +Maintainer: scripts team +Build-Depends: debhelper (>= 7.0.50~), config-package-dev, munin-node +Standards-Version: 3.9.4 +Homepage: http://scripts.mit.edu/ + +Package: scripts-vm-host +Architecture: all +Depends: ${misc:Depends}, + apticron, + build-essential, + bwm-ng, + bzip2, + emacs23-nox, + ethtool, + git, + htop, + i2c-tools, + ipmitool, + kpartx, + lm-sensors, + memtest86+, + memtest86, + mii-diag, + molly-guard, + mtr-tiny, + nbd-client, + nbd-server, + ntp, + ntpdate, + rlwrap, + smartmontools, + strace, + tcpdump, + tree, + vim, + xen-linux-system, + debathena-clients, + debathena-ssh-server-config, + sudo, + conserver-client, + conserver-server, + munin-node, + subversion, + screen, + scripts-syslog-ng-config, + sysstat, +Description: Configures a machine to be a scripts VM host + Configures a machine to be a scripts VM host, installing all + appropriate dependencies. Index: branches/fc20-dev/host/debian/scripts-vm-host/debian/copyright =================================================================== --- branches/fc20-dev/host/debian/scripts-vm-host/debian/copyright (revision 2523) +++ branches/fc20-dev/host/debian/scripts-vm-host/debian/copyright (revision 2523) @@ -0,0 +1,22 @@ +This work was packaged for Debian by: + + Quentin Smith on Wed, 08 Jun 2011 23:22:31 -0400 + +It was downloaded from: + + http://scripts.mit.edu + +Copyright: + + Copyright (C) 2011 Quentin Smith + +License: + + GPLv2+ + +The Debian packaging is: + + Copyright (C) 2011 Quentin Smith + +and is licensed under the GPL version 2 +see "/usr/share/common-licenses/GPL-2". Index: branches/fc20-dev/host/debian/scripts-vm-host/debian/rules =================================================================== --- branches/fc20-dev/host/debian/scripts-vm-host/debian/rules (revision 2523) +++ branches/fc20-dev/host/debian/scripts-vm-host/debian/rules (revision 2523) @@ -0,0 +1,3 @@ +#!/usr/bin/make -f +%: + dh $@ --with config-package Index: branches/fc20-dev/host/debian/scripts-vm-host/debian/scripts-vm-host.install =================================================================== --- branches/fc20-dev/host/debian/scripts-vm-host/debian/scripts-vm-host.install (revision 2523) +++ branches/fc20-dev/host/debian/scripts-vm-host/debian/scripts-vm-host.install (revision 2523) @@ -0,0 +1,3 @@ +gitconfig /etc +conserver.cf.divert /etc/conserver +conserver-sudoers /etc/sudoers.d Index: branches/fc20-dev/host/debian/scripts-vm-host/debian/scripts-vm-host.postinst =================================================================== --- branches/fc20-dev/host/debian/scripts-vm-host/debian/scripts-vm-host.postinst (revision 2523) +++ branches/fc20-dev/host/debian/scripts-vm-host/debian/scripts-vm-host.postinst (revision 2523) @@ -0,0 +1,47 @@ +#!/bin/sh +# postinst script for #PACKAGE# +# +# see: dh_installdeb(1) + +set -e + +# summary of how this script can be called: +# * `configure' +# * `abort-upgrade' +# * `abort-remove' `in-favour' +# +# * `abort-remove' +# * `abort-deconfigure' `in-favour' +# `removing' +# +# for details, see http://www.debian.org/doc/debian-policy/ or +# the debian-policy package + + +case "$1" in + configure) + debconf-set-selections <&2 + exit 1 + ;; +esac + +# dh_installdeb will replace this with shell code automatically +# generated by other debhelper scripts. + +#DEBHELPER# + +exit 0 Index: branches/fc20-dev/host/debian/scripts-vm-host/debian/scripts-vm-host.transform =================================================================== --- branches/fc20-dev/host/debian/scripts-vm-host/debian/scripts-vm-host.transform (revision 2523) +++ branches/fc20-dev/host/debian/scripts-vm-host/debian/scripts-vm-host.transform (revision 2523) @@ -0,0 +1,1 @@ +/etc/munin/munin-node.conf.scripts debian/transform_munin-node.conf.scripts Index: branches/fc20-dev/host/debian/scripts-vm-host/debian/transform_munin-node.conf.scripts =================================================================== --- branches/fc20-dev/host/debian/scripts-vm-host/debian/transform_munin-node.conf.scripts (revision 2523) +++ branches/fc20-dev/host/debian/scripts-vm-host/debian/transform_munin-node.conf.scripts (revision 2523) @@ -0,0 +1,9 @@ +#!/bin/sh + +cat +cat < /etc/apt/sources.list.d/debathena.list +deb http://debathena.mit.edu/apt squeeze debathena debathena-config debathena-system openafs +deb-src http://debathena.mit.edu/apt squeeze debathena debathena-config debathena-system openafs +EOF + +# add scripts repos to /etc/apt/sources.list.d + cat < /etc/apt/sources.list.d/scripts.list +deb http://web.mit.edu/scripts/apt stable main +deb-src http://web.mit.edu/scripts/apt stable main +EOF + +# add scripts apt repo key + +# install ~/.k5login +# clone the xen config (/etc/xen) + git clone -b squeeze ssh://scripts@scripts.mit.edu/mit/scripts/git/xen.git /etc/xen + +# Install scripts-vm-host + aptitude update + aptitude install scripts-vm-host + +# install host keytab + cp $keytab /etc/krb5.keytab + k5srvutil change + k5srvutil delold + +# Configure exim4 to use smarthost (outgoing.mit.edu), no local mail +dpkg-reconfigure exim4-config Index: branches/fc20-dev/locker/bin/cronload =================================================================== --- branches/fc20-dev/locker/bin/cronload (revision 2523) +++ branches/fc20-dev/locker/bin/cronload (revision 2523) @@ -0,0 +1,41 @@ +#!/bin/sh + +usage="Usage \"$0 [-l lockername] [-h] crontab\"" +while getopts "l:h" options; do + case $options in + l ) lname=$OPTARG;; + h ) echo "$usage"; exit 0;; + * ) echo "$usage"; exit 1;; + esac +done +shift `expr $OPTIND - 1` +if [ -z "$1" ]; then + echo "$usage" + exit 1 +fi + +echo "This program should print your new crontab below." +echo "If it does not do so, something is wrong." +echo "Feel free to contact scripts@mit.edu for assistance." +echo + + +cwd=`pwd` +if [ -z "$lname" ]; then + lname=`perl -e "\\\$temp = \"$cwd\"; \\\$temp =~ /\\\/([^\\\/]+)\\\/cron_scripts/; print \\\$1"` +fi +if [ -z "$lname" ]; then + echo "ERROR: Could not detect locker name. Make sure to run" + echo "cronload from within /mit/lockername/cron_scripts/" + echo "(or pass the -l lockername option)" + exit 1 +fi +athrun scripts scripts-ssh "$lname" /usr/local/bin/cronload "$1" "$cwd" 2>/dev/null +if ! grep -q "^MAILTO=" "$1"; then + echo "WARNING: You have no MAILTO= variable set. This means any" + echo "cron errors will go to $lname@scripts.mit.edu (your mail_scripts" + echo "account), which is almost certainly not what you want!" + echo "Please add a MAILTO= line, e.g., MAILTO=${EMAIL:-${ATHENA_USER:-$USER}@mit.edu}," + echo "to your crontab. If you do not want to receive errors, set" + echo 'MAILTO="".' +fi Index: branches/fc20-dev/locker/bin/crontab =================================================================== --- branches/fc20-dev/locker/bin/crontab (revision 2523) +++ branches/fc20-dev/locker/bin/crontab (revision 2523) @@ -0,0 +1,35 @@ +# scripts.mit.edu sample crontab as of SCRIPTS_DATE +# To load this crontab, run "cronload crontab" in your cron_scripts directory +# (On athena, you must run "add scripts" before cronload) + +# This line sets a reasonable default path +PATH=/mit/SCRIPTS_USER/cron_scripts:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin + +# This line mails the STDOUT and STDERR of every cron script to a person +# (can be useful for debugging) +# You can always redirect the output of individual commands to /dev/null +MAILTO="ATHENA_USER@mit.edu" +# If you do not want to receive any mail from cron, use the line below instead +#MAILTO="" + +# Add your cron lines here: + + +# Here's a reminder of the cron line format: + +# * * * * * command to be executed +# - - - - - +# | | | | | +# | | | | ----- day of week (0 - 6) (Sunday=0) +# | | | ------- month (1 - 12) +# | | --------- day of month (1 - 31) +# | ----------- hour (0 - 23) +# ------------- min (0 - 59) + +# For example, the following cron line would write "Work harder!" to STDOUT +# every 20 minutes from 8am - 5pm on weekdays +# (and this message would thus be e-mailed to the address above on each run) + +# 0,20,40 8-17 * * 1-5 echo "Work harder!" + +# See http://en.wikipedia.org/wiki/Cron (or google for crontab) for more info Index: branches/fc20-dev/locker/bin/disable-scripts-test =================================================================== --- branches/fc20-dev/locker/bin/disable-scripts-test (revision 2523) +++ branches/fc20-dev/locker/bin/disable-scripts-test (revision 2523) @@ -0,0 +1,5 @@ +#!/bin/sh +echo Removing iptables rules. +iptables -t nat -D OUTPUT -d 18.181.0.46 -j DNAT --to-destination 18.181.0.229 +iptables -t nat -D OUTPUT -d 18.181.0.43 -j DNAT --to-destination 18.181.0.229 +iptables -t nat -D OUTPUT -d 18.181.0.50 -j DNAT --to-destination 18.181.0.229 Index: branches/fc20-dev/locker/bin/enable-scripts-test =================================================================== --- branches/fc20-dev/locker/bin/enable-scripts-test (revision 2523) +++ branches/fc20-dev/locker/bin/enable-scripts-test (revision 2523) @@ -0,0 +1,5 @@ +#!/bin/sh +echo Adding iptables rules. +iptables -t nat -A OUTPUT -d 18.181.0.46 -j DNAT --to-destination 18.181.0.229 +iptables -t nat -A OUTPUT -d 18.181.0.43 -j DNAT --to-destination 18.181.0.229 +iptables -t nat -A OUTPUT -d 18.181.0.50 -j DNAT --to-destination 18.181.0.229 Index: branches/fc20-dev/locker/bin/firefox-test =================================================================== --- branches/fc20-dev/locker/bin/firefox-test (revision 2523) +++ branches/fc20-dev/locker/bin/firefox-test (revision 2523) @@ -0,0 +1,5 @@ +#!/bin/sh +attach -q scripts +LD_PRELOAD=/mit/scripts/scripts-test/@sys/scripts-test-preload.so +export LD_PRELOAD +exec firefox -no-remote Index: branches/fc20-dev/locker/bin/fix-php-ini =================================================================== --- branches/fc20-dev/locker/bin/fix-php-ini (revision 2523) +++ branches/fc20-dev/locker/bin/fix-php-ini (revision 2523) @@ -0,0 +1,20 @@ +#!/bin/sh + +# This script is meant to help people who have somehow lost their +# php.ini files. It is meant to be run in the top level directory +# of an application once a reasonable php.ini file has been placed +# there, and will make the symlinks to it in all child directories. + +if [ -f php.ini ]; then + echo "Creating php.ini symlinks in child directories..." + athrun scripts gfind . -mindepth 1 -type d \( -not -name .svn -not -name .git -or -not -prune \) -exec sh -c 'ln -sf "`echo "$1" | sed '\''s,[^/],,g; s,/,../,g'\''`php.ini" "$1/"' -- {} \; + echo "Done!" +else + echo "There is no php.ini file in this directory. You should first" + echo "put a valid php.ini file in the top level directory of your" + echo "application, then change to that directory, and then run this" + echo "script to make the symlinks to your php.ini file from all the" + echo "child directories." + exit 1 +fi + Index: branches/fc20-dev/locker/bin/fix-php-ini-scripts =================================================================== --- branches/fc20-dev/locker/bin/fix-php-ini-scripts (revision 2523) +++ branches/fc20-dev/locker/bin/fix-php-ini-scripts (revision 2523) @@ -0,0 +1,20 @@ +#!/bin/sh + +# This script is meant to help people who have somehow lost their +# php.ini files. It is meant to be run in the top level directory +# of an application once a reasonable php.ini file has been placed +# there, and will make the symlinks to it in all child directories. + +if [ -f php.ini ]; then + echo "Creating php.ini symlinks in child directories..." + find . -mindepth 1 -type d -exec sh -c 'ln -sf "`echo "$1" | sed '\''s,[^/],,g; s,/,../,g'\''`php.ini" "$1/"' -- {} \; + echo "Done!" +else + echo "There is no php.ini file in this directory. You should first" + echo "put a valid php.ini file in the top level directory of your" + echo "application, then change to that directory, and then run this" + echo "script to make the symlinks to your php.ini file from all the" + echo "child directories." + exit 1 +fi + Index: branches/fc20-dev/locker/bin/for-each-server =================================================================== --- branches/fc20-dev/locker/bin/for-each-server (revision 2523) +++ branches/fc20-dev/locker/bin/for-each-server (revision 2523) @@ -0,0 +1,5 @@ +#!/bin/sh + +for server in `finger @scripts.mit.edu | sed -n -e "1,5d" -e "s/ -> \([^:]*\):.*/\1/p" | sort -u`; do + ssh "$server" "$@" +done Index: branches/fc20-dev/locker/bin/fssar =================================================================== --- branches/fc20-dev/locker/bin/fssar (revision 2523) +++ branches/fc20-dev/locker/bin/fssar (revision 2523) @@ -0,0 +1,3 @@ +#!/bin/sh + +exec athrun consult fsr sa . "$@" Index: branches/fc20-dev/locker/bin/gfind =================================================================== --- branches/fc20-dev/locker/bin/gfind (revision 2523) +++ branches/fc20-dev/locker/bin/gfind (revision 2523) @@ -0,0 +1,7 @@ +#!/bin/sh + +if find / -maxdepth 0 >/dev/null >&2; then + exec find "$@" +else + exec athrun gnu gfind "$@" +fi Index: branches/fc20-dev/locker/bin/gtar =================================================================== --- branches/fc20-dev/locker/bin/gtar (revision 2523) +++ branches/fc20-dev/locker/bin/gtar (revision 2523) @@ -0,0 +1,8 @@ +#!/bin/sh + +gnu=`tar --version 2>/dev/null | grep -i gnu` +if [ "$gnu" != "" ]; then + exec tar "$@" +else + exec athrun gnu gtar "$@" +fi Index: branches/fc20-dev/locker/bin/procmailrc =================================================================== --- branches/fc20-dev/locker/bin/procmailrc (revision 2523) +++ branches/fc20-dev/locker/bin/procmailrc (revision 2523) @@ -0,0 +1,3 @@ +:0 +* +! SCRIPTS_USER@mit.edu Index: branches/fc20-dev/locker/bin/scripts =================================================================== --- branches/fc20-dev/locker/bin/scripts (revision 2523) +++ branches/fc20-dev/locker/bin/scripts (revision 2523) @@ -0,0 +1,30 @@ +#!/bin/sh + +choices () { + echo 'scripts-start Begin a Quick-Start autoinstall (wikis, blogs, etc.)' + echo 'signup-web Enable the web scripts service' + echo 'signup-cron Enable the cron scripts service' + echo 'signup-mail Enable the mail scripts service' + echo 'signup-sql Sign up for a sql.mit.edu account' +} + +nchoices=`choices | wc -l` + +echo +echo "Welcome to scripts.mit.edu. Which service would you like to use?" +echo +choices | sed 's/^[^ ]* //' | cat -n +echo +printf "Please enter a number 1-%d: " "$nchoices" +read num +echo + +attach scripts 2>/dev/null +choice=`choices | sed -n "$num { s/ .*$//; p; }"` +if [ -n "$choice" ]; then + . "/mit/scripts/bin$scriptsdev/$choice" +else + echo "ERROR:" + echo "You must enter a number 1 through $nchoices." + exit 1 +fi Index: branches/fc20-dev/locker/bin/scripts-django =================================================================== --- branches/fc20-dev/locker/bin/scripts-django (revision 2523) +++ branches/fc20-dev/locker/bin/scripts-django (revision 2523) @@ -0,0 +1,7 @@ +#!/bin/sh + +sname="Django" +deploy="django" +prompt_username=1 +create_scripts_dir=1 +. /mit/scripts/deploy$scriptsdev/bin/onathena Index: branches/fc20-dev/locker/bin/scripts-gallery2 =================================================================== --- branches/fc20-dev/locker/bin/scripts-gallery2 (revision 2523) +++ branches/fc20-dev/locker/bin/scripts-gallery2 (revision 2523) @@ -0,0 +1,8 @@ +#!/bin/sh + +sname="Gallery2" +deploy="gallery2" +prompt_username=1 +create_dir=1 + +. /mit/scripts/deploy$scriptsdev/bin/onathena Index: branches/fc20-dev/locker/bin/scripts-git =================================================================== --- branches/fc20-dev/locker/bin/scripts-git (revision 2523) +++ branches/fc20-dev/locker/bin/scripts-git (revision 2523) @@ -0,0 +1,9 @@ +#!/bin/sh + +sname="git repository" +deploy="git" +create_scripts_dir=1 +requires_sql=0 +prompt_username=1 +prompt_password=0 +. /mit/scripts/deploy$scriptsdev/bin/onathena Index: branches/fc20-dev/locker/bin/scripts-joomla =================================================================== --- branches/fc20-dev/locker/bin/scripts-joomla (revision 2523) +++ branches/fc20-dev/locker/bin/scripts-joomla (revision 2523) @@ -0,0 +1,5 @@ +#!/bin/sh + +sname="Joomla" +deploy="joomla" +. /mit/scripts/deploy$scriptsdev/bin/onathena Index: branches/fc20-dev/locker/bin/scripts-mediawiki =================================================================== --- branches/fc20-dev/locker/bin/scripts-mediawiki (revision 2523) +++ branches/fc20-dev/locker/bin/scripts-mediawiki (revision 2523) @@ -0,0 +1,7 @@ +#!/bin/sh + +sname="MediaWiki" +deploy="mediawiki" +prompt_username=1 +wizard="mediawiki" +. /mit/scripts/deploy$scriptsdev/bin/onathena Index: branches/fc20-dev/locker/bin/scripts-phpbb =================================================================== --- branches/fc20-dev/locker/bin/scripts-phpbb (revision 2523) +++ branches/fc20-dev/locker/bin/scripts-phpbb (revision 2523) @@ -0,0 +1,5 @@ +#!/bin/sh + +sname="phpBB" +deploy="phpbb" +. /mit/scripts/deploy$scriptsdev/bin/onathena Index: branches/fc20-dev/locker/bin/scripts-rails =================================================================== --- branches/fc20-dev/locker/bin/scripts-rails (revision 2523) +++ branches/fc20-dev/locker/bin/scripts-rails (revision 2523) @@ -0,0 +1,9 @@ +#!/bin/sh + +sname="Ruby on Rails" +deploy="rails" +prompt_password=0 +# The following is sort of a lie. Rails wants three databases (dev/test/prod), +# so the rails script will handle it manually. +requires_sql=0 +. /mit/scripts/deploy$scriptsdev/bin/onathena Index: branches/fc20-dev/locker/bin/scripts-remove =================================================================== --- branches/fc20-dev/locker/bin/scripts-remove (revision 2523) +++ branches/fc20-dev/locker/bin/scripts-remove (revision 2523) @@ -0,0 +1,162 @@ +#!/bin/bash + +if [ "$scriptsdev" != "" -a "$scriptsdev" != "dev" ]; then + echo "ERROR:" + echo "The \$scriptsdev variable is set to an invalid value." + echo "(The variable should not be set.)" + echo "Please contact scripts@mit.edu." +fi + +sshrun() { + athrun scripts scripts-ssh "$lname" "/mit/scripts/$@" 2>/dev/null +} + +checksqlpass() { + errors=`sshrun "sql/bin$scriptsdev/test-password"` + if [ "$errors" != "" ]; then + if [ "$1" -eq 1 ]; then + rm -f "$lroot/.sql/my.cnf" + fi + echo + echo "ERROR:" + printf "$2" + exit 1 + fi +} +attach scripts sql 2>/dev/null + +echo "Welcome to the scripts.mit.edu uninstaller. This program will" +echo "help you cleanly remove software that you have auto-installed." +echo +echo "Are you removing an installation from:" +echo "1. Your personal Athena account" +echo "2. A locker that you control (a club, a course, etc.)" +echo "If you do not understand this question, you should answer 1." +printf "Please enter either 1 or 2: " +read whofor +if [ "$whofor" = 1 ]; then + lname=$USER + lroot=$HOME +elif [ "$whofor" = 2 ]; then + echo + echo "Please enter the name of the selected locker below." + echo "(For the locker /mit/sipb, you would enter sipb.)" + read lname + lroot="/mit/$lname" +else + echo + echo "ERROR:" + echo "You must select either 1 or 2." + exit 1 +fi +attach "$lname" 2>/dev/null + +echo +echo "When you installed the software, you chose a URL" +echo "that starts with http://$lname.scripts.mit.edu/" +echo "(for software installed after March 2009) or" +echo "http://scripts.mit.edu/~$lname/ (for software" +echo "installed before then)." +echo "Please enter the new-style full URL where this" +echo "software was installed. (This should correspond" +echo "to a directory in /mit/$lname/web_scripts/.)" +printf "%s" "URL: http://$lname.scripts.mit.edu/" +read addrend + +addrend=`perl -0e 'print $ARGV[0] =~ /^([\w\/-]*[\w-])\/*$/' -- "$addrend"` +if [ "$addrend" = "" ]; then + echo + echo "ERROR:" + echo "You must enter one or more characters after mit.edu/" + echo "The completed address must only contain a-z, 0-9, and /." + exit 1 +fi + +if [ ! -d "$lroot/web_scripts/$addrend" ]; then + echo + echo "ERROR:" + echo "The directory $lroot/web_scripts/$addrend" + echo "does not exist. Please make sure that this is the" + echo "correct installation directory, and try again, or" + echo "contact scripts@mit.edu for assistance." + exit 1 +fi + +echo + +sqlinfo=`sshrun "sql/bin$scriptsdev/get-password"` +if [ "$sqlinfo" = "" ]; then + echo + echo "You have a MySQL account but you do not have a .my.cnf file." + echo "If you do not remember your MySQL account password, you can change it" + echo "at http://sql.mit.edu using MIT certificates." + printf "Please type your MySQL password and press [enter]: " + stty -echo + read sqlpass + stty echo + echo + sqlhost="sql.mit.edu" + sqluser=$lname + . "/mit/scripts/sql/bin$scriptsdev/save-password" + checksqlpass 1 'The MySQL password that you typed appears to be incorrect.\n' + echo + echo "OK. Continuing with the uninstaller..." +else + checksqlpass 0 'The MySQL login information in your .my.cnf file\nappears to be incorrect.\n' +fi + +sqldb=`sshrun "sql/bin$scriptsdev/get-next-database" "$addrend"` +if [ "$sqldb" != "${addrend}1" -a "$sqldb" != "$addrend" ]; then + echo + echo "ERROR:" + echo "The auto-uninstaller was unable to find the appropriate" + echo "database to drop. Please examine the installation to" + echo "find the database it uses, drop the database from" + echo "http://sql.mit.edu/, and manually remove the $addrend" + echo "directory (or re-run the auto-installer). Contact" + echo "scripts@mit.edu if you need assistance." + exit 1 +fi + +echo "Removing files. Please wait..." +echo "(This may take several seconds for large software.)" +if rm -rf "$lroot/web_scripts/$addrend"; then + echo "The directory $lroot/web_scripts/$addrend" + echo "was successfully removed." + if [ -d "$lroot/OldFiles/web_scripts/$addrend" ]; then + echo "A one-day-old backup of the installation is" + echo "available from $lroot/OldFiles/web_scripts/$addrend". + fi +else + echo "ERROR:" + echo "The directory $lroot/web_scripts/$addrend" + echo "could not be removed. Please ensure that you have" + echo "access to this directory and try again, or" + echo "contact scripts@mit.edu for assistance." + exit 1 +fi + +echo +if [ "$sqldb" = "${addrend}1" ]; then + sqldb="$lname+$addrend" + dropped=`sshrun "sql/bin$scriptsdev/drop-database" "$sqldb"` + if [ "$dropped" ]; then + echo "The database $sqldb" + echo "was successfully removed." + attach sql 2>/dev/null + if [ -f "/mit/sql/backup/$lname/$sqldb.sql.gz" ]; then + echo "A one-day-old backup of your SQL database is" + echo "available in /mit/sql/backup/$lname". + fi + else + echo "ERROR:" + echo "The database $lname+$addrend" + echo "could not be automatically removed. You can" + echo "try removing it from http://sql.mit.edu/," + echo "or you can contact sql@mit.edu for assistance." + exit 1 + fi +fi +echo +echo "The installation in http://$lname.scripts.mit.edu/$addrend" +echo "has been successfully uninstalled." Index: branches/fc20-dev/locker/bin/scripts-ssh =================================================================== --- branches/fc20-dev/locker/bin/scripts-ssh (revision 2523) +++ branches/fc20-dev/locker/bin/scripts-ssh (revision 2523) @@ -0,0 +1,19 @@ +#!/bin/bash + +lname=$1 +shift + +exec ssh \ + -o GSSAPIAuthentication=yes \ + -o GSSAPIDelegateCredentials=no \ + -o PreferredAuthentications=gssapi-with-mic \ + -o ForwardX11=no \ + -o GlobalKnownHostsFile=/afs/athena.mit.edu/contrib/scripts/etc/known_hosts \ + -o UserKnownHostsFile=/dev/null \ + -t \ + scripts.mit.edu -l "$lname" "$(printf "''%q " "$@")" + +# ssh gets quoting wrong, so we do it ourself with printf "%q ". +# Except bash 2 gets printf "%q " wrong for empty arguments, so we use +# printf "''%q " instead. +# Isn't software fun? Index: branches/fc20-dev/locker/bin/scripts-start =================================================================== --- branches/fc20-dev/locker/bin/scripts-start (revision 2523) +++ branches/fc20-dev/locker/bin/scripts-start (revision 2523) @@ -0,0 +1,34 @@ +#!/bin/sh + +choices () { + echo 'mediawiki MediaWiki' + echo 'wordpress WordPress' + echo 'gallery2 Gallery2' + echo 'phpbb phpBB' + echo 'git Git repository' + echo 'trac Trac' + echo 'turbogears TurboGears' + echo 'django Django' + echo 'rails Ruby on Rails' +} + +nchoices=`choices | wc -l` + +echo +echo "What piece of software would you like to install?" +echo +choices | sed 's/^[^ ]* //' | cat -n +echo +printf "Please enter a number 1-%d: " "$nchoices" +read num +echo + +attach scripts 2>/dev/null +choice=`choices | sed -n "$num { s/ .*$//; p; }"` +if [ -n "$choice" ]; then + . "/mit/scripts/bin$scriptsdev/scripts-$choice" +else + echo "ERROR:" + echo "You must enter a number 1 through $nchoices." + exit 1 +fi Index: branches/fc20-dev/locker/bin/scripts-trac =================================================================== --- branches/fc20-dev/locker/bin/scripts-trac (revision 2523) +++ branches/fc20-dev/locker/bin/scripts-trac (revision 2523) @@ -0,0 +1,6 @@ +#!/bin/sh + +sname="Trac" +deploy="trac" +prompt_password=0 +. /mit/scripts/deploy$scriptsdev/bin/onathena Index: branches/fc20-dev/locker/bin/scripts-turbogears =================================================================== --- branches/fc20-dev/locker/bin/scripts-turbogears (revision 2523) +++ branches/fc20-dev/locker/bin/scripts-turbogears (revision 2523) @@ -0,0 +1,7 @@ +#!/bin/sh + +sname="TurboGears" +deploy="turbogears" +prompt_password=0 +create_scripts_dir=1 +. /mit/scripts/deploy$scriptsdev/bin/onathena Index: branches/fc20-dev/locker/bin/scripts-wordpress =================================================================== --- branches/fc20-dev/locker/bin/scripts-wordpress (revision 2523) +++ branches/fc20-dev/locker/bin/scripts-wordpress (revision 2523) @@ -0,0 +1,7 @@ +#!/bin/sh + +sname="WordPress" +deploy="wordpress" +prompt_password=0 +wizard="wordpress" +. /mit/scripts/deploy$scriptsdev/bin/onathena Index: branches/fc20-dev/locker/bin/signup =================================================================== --- branches/fc20-dev/locker/bin/signup (revision 2523) +++ branches/fc20-dev/locker/bin/signup (revision 2523) @@ -0,0 +1,18 @@ +#!/bin/sh + +if [ "$1" = "web" ]; then + athrun scripts signup-web +elif [ "$1" = "cron" ]; then + athrun scripts signup-cron +elif [ "$1" = "mail" ]; then + athrun scripts signup-mail +elif [ "$1" = "sql" ]; then + athrun scripts signup-sql +else + echo "Scripts has multiple services you can sign up for:" + echo " signup-web Web script hosting" + echo " signup-cron Scheduled jobs" + echo " signup-mail Mail scripts" + echo " signup-sql MySQL hosting" + exit 1 +fi Index: branches/fc20-dev/locker/bin/signup-cron =================================================================== --- branches/fc20-dev/locker/bin/signup-cron (revision 2523) +++ branches/fc20-dev/locker/bin/signup-cron (revision 2523) @@ -0,0 +1,20 @@ +#!/bin/sh + +attach -q scripts +. /mit/scripts/bin$scriptsdev/signup-minimal + +if [ ! -d "/mit/$lname/cron_scripts/" ]; then + mkdir -p /mit/$lname/cron_scripts + fs sa /mit/$lname/cron_scripts system:anyuser l + fs sa /mit/$lname/cron_scripts system:authuser none + fs sa /mit/$lname/cron_scripts daemon.scripts write + DATE=`date` + signupuser=`echo "$principal" | sed 's/[/@].*$//'` + sed '/SCRIPTS_USER/s//'"$lname"'/g' /mit/scripts/bin$scriptsdev/crontab | sed '/ATHENA_USER/s//'"$signupuser"'/g' | sed '/SCRIPTS_DATE/s//'"$DATE"'/g' > /mit/$lname/cron_scripts/crontab + echo "By default, output from cron jobs for the $lname locker will be mailed" + echo "to $signupuser@mit.edu. You should edit /mit/$lname/cron_scripts/crontab" + echo "to change this and set up your cron jobs." + success "the cron script service" "The directory /mit/$lname/cron_scripts has been created." +else + success "the cron script service" "The directory /mit/$lname/cron_scripts already exists." +fi Index: branches/fc20-dev/locker/bin/signup-mail =================================================================== --- branches/fc20-dev/locker/bin/signup-mail (revision 2523) +++ branches/fc20-dev/locker/bin/signup-mail (revision 2523) @@ -0,0 +1,19 @@ +#!/bin/sh + +attach -q scripts +. /mit/scripts/bin$scriptsdev/signup-minimal + +if [ ! -d "/mit/$lname/mail_scripts/" ]; then + mkdir -p /mit/$lname/mail_scripts + fs sa /mit/$lname/mail_scripts system:anyuser l + fs sa /mit/$lname/mail_scripts system:authuser none + fs sa /mit/$lname/mail_scripts daemon.scripts read + signupuser=`echo "$principal" | sed 's/[/@].*$//'` + sed /SCRIPTS_USER/s//$signupuser/ /mit/scripts/bin/procmailrc > /mit/$lname/mail_scripts/procmailrc + echo "By default, mail sent to $lname@scripts.mit.edu will be forwarded to" + echo "$signupuser@mit.edu. You should edit /mit/$lname/mail_scripts/procmailrc" + echo "to change this and customize your procmail configuration." + success "the mail script service" "The directory /mit/$lname/mail_scripts has been created." +else + success "the mail script service" "The directory /mit/$lname/mail_scripts already exists." +fi Index: branches/fc20-dev/locker/bin/signup-minimal =================================================================== --- branches/fc20-dev/locker/bin/signup-minimal (revision 2523) +++ branches/fc20-dev/locker/bin/signup-minimal (revision 2523) @@ -0,0 +1,126 @@ +#!/bin/bash + +if [ "$initlname" = "" ]; then + if [ "$lname" = "" ]; then + initlname=0 + else + initlname=1 + fi +fi + +if type wget >/dev/null 2>/dev/null; then + WGET=wget +else + WGET="athrun gnu wget" +fi + +if [ "$lname" = "" ]; then + if [ "$1" = "" ]; then + echo + echo "Would you like to sign up:" + echo "1. Your personal Athena account" + echo "2. A locker that you control (a club, a course, etc)" + echo "If you do not understand this question, you should answer '1'." + printf "Please enter either '1' or '2' (without quotes): " + read whofor + if [ "$whofor" = 1 ]; then + lname="${ATHENA_USER:-$USER}" + elif [ "$whofor" = 2 ]; then + echo + echo "OK. A locker of your choice that you control will be signed up." + echo "Please enter the name of the selected locker below." + echo "(For the locker /mit/sipb, you would enter sipb)." + printf "Locker name: " + read lname + else + echo + echo "ERROR:" + echo "You must select either '1' or '2'." + exit 1 + fi + while true; do + if attach "$lname"; then + break + fi + echo "$lname is not a valid locker name." + printf "Locker name: " + read lname + done + else + lname="$1" + fi +fi +lroot="/mit/$lname" + +attach "$lname" 2>/dev/null + +ans=`$WGET -q -O- "http://scripts.mit.edu/~signup/fsla.php/mit/$lname"` +if [ "$ans" != "0" ]; then + echo + echo "ERROR:" + echo "The scripts servers cannot verify the permissions of the locker <$lname>." + echo "This is probably because your locker is not publicly listable." + echo "You can remedy this signup problem and make your locker publicly" + echo "listable by running \"fs setacl /mit/$lname system:anyuser l\"" + echo "(that's a lowercase L at the end)." + echo "" + echo "NOTE: This will make it possible for the public (including anyone" + echo "viewing http://web.mit.edu/$lname) to see the names of your files" + echo "and the list of people who have access to them, though it will not" + echo "cause the contents of your files to be publicly readable. If you" + echo "are unwilling to have your locker listable by the public, please" + echo "contact scripts@mit.edu for information about other ways to work" + echo "around the problem, or see http://scripts.mit.edu/faq/122 for more" + echo "detailed information." + exit 1 +fi + +principal=`klist -5 | sed -n 's/^Default principal: // p'` +ans=`$WGET -q -O- "http://scripts.mit.edu/~signup/admof.php/$lname/$principal"` +if [ "$ans" != "yes" ]; then + afsuser=`echo "$principal" | sed 's/@ATHENA.MIT.EDU$//'` + echo + echo "ERROR:" + echo "It appears as though you are not an administrator of the locker <$lname>." + echo "Try running \"fs setacl /mit/$lname $afsuser all\" and starting over." + echo "Contact scripts@mit.edu if you are unable to solve the problem." + exit 1 +fi + +mkdir -p "/mit/$lname/.scripts-signup" + +if [ ! -d "/mit/$lname/.scripts-signup" ]; then + echo + echo "ERROR:" + echo "It appears as though you do not have write access to the locker <$lname>." + echo "Contact scripts@mit.edu if you are unable to solve the problem." + exit 1 +fi + +ans=`$WGET -q -O- "http://scripts.mit.edu/~signup/$lname"` +rmdir "/mit/$lname/.scripts-signup" + +if [ "$ans" = "done" ]; then + # nscd caches account nonexistence with a 5-second TTL. + # (LDAP updates are more or less instant.) + # Somehow, the server can wait up to 10 seconds... + echo "Creating scripts.mit.edu account for $lname..." + sleep 10 +fi + +if [ "$ans" != "done" ] && [ "$ans" != "username already taken" ]; then + echo "ERROR:" + echo "Signup reported the following error: \"$ans\"." + echo "Contact scripts@mit.edu for assistance." + exit 1 +fi + +success() { + if [ "$initlname" -eq 0 ]; then + echo + echo "== SUCCESS ==" + echo "$lname is now signed up for $1." + echo "$2" + echo + fi +} Index: branches/fc20-dev/locker/bin/signup-sql =================================================================== --- branches/fc20-dev/locker/bin/signup-sql (revision 2523) +++ branches/fc20-dev/locker/bin/signup-sql (revision 2523) @@ -0,0 +1,119 @@ +#!/bin/sh + +checkfailed() { + if [ -f "$lroot/web_scripts/$addrend/.failed" ]; then + rm -f $lroot/web_scripts/$addrend/.failed + exit 1 + fi +} + +sshrun() { + athrun scripts scripts-ssh "$lname" "/mit/scripts/$@" 2>/dev/null +} + +vsshrun() { + athrun scripts scripts-ssh "$lname" "/mit/scripts/$@" +} + +checksqlpass() { + errors=`sshrun "sql/bin$scriptsdev/test-password"` + if [ "$errors" != "" ]; then + if [ "$1" -eq 1 ]; then + rm -f $lroot/.sql/my.cnf + fi + echo + echo ERROR: + printf "$2" + printf "$3" + exit + fi +} + +echo +echo == Welcome to the sql.mit.edu signup process == +echo + +echo "For documentation, including a link to the Athena rules of use," +echo "see ." +echo +echo Please report problems with this signup process to sql@mit.edu. +echo +echo Are you performing this signup for: +echo 1. Your personal Athena account +echo 2. A locker that you control \(a club, a course, etc\) +echo "If you do not understand this question, you should answer '1'". +printf "Please enter either '1' or '2' (without quotes): " +read whofor +if [ "$whofor" -eq 1 ]; then + lname=$USER + lroot=$HOME +elif [ "$whofor" -eq 2 ]; then + echo + echo OK. A MySQL account will be created for a locker of your choice that + echo you control. Please enter the name of the selected locker below. + echo "(For the locker /mit/sipb, you would enter sipb)." + while true; do + printf "Locker name: " + read lname + if attach "$lname"; then + break + fi + echo "$lname is not a valid locker name." + done + lroot="/mit/$lname" +else + echo + echo ERROR: + echo You must select either '1' or '2'. + exit 1 +fi + +# Users need to sign up for scripts.mit.edu so that they can +# authenticate themselves to sql.mit.edu +attach scripts +. /mit/scripts/bin$scriptsdev/signup-web + +if [ "$requires_sql" = "" ]; then + requires_sql=1 +fi + +if [ ! -f "$lroot/.my.cnf" ]; then + mkdir -p $lroot/.sql + fs sa $lroot/.sql system:anyuser none + fs sa $lroot/.sql system:authuser none + fs sa $lroot/.sql daemon.scripts write + fs sa $lroot/.sql daemon.sql write + ln -nfs .sql/my.cnf $lroot/.my.cnf +fi + +if [ "$requires_sql" -eq 1 ]; then + sqlinfo=`sshrun "sql/bin$scriptsdev/get-password"` + if [ "$sqlinfo" = "" ]; then + echo + echo You already have a MySQL account but you do not have a .my.cnf file. + echo If you do not remember your MySQL account password, you can change it + echo at http://sql.mit.edu using MIT certificates. + printf "Please type your MySQL password and press [enter]: " + stty -echo + read sqlpass + stty echo + echo + sqlhost="sql.mit.edu" + sqluser=$lname + . /mit/scripts/sql/bin$scriptsdev/save-password + checksqlpass 1 'The MySQL password that you typed appears to be incorrect.\n' '' + echo + echo OK. Continuing with the install... + else + checksqlpass 0 'The MySQL login information in your .my.cnf file\n' 'appears to be incorrect.\n' + fi +fi + +echo +echo == Setup complete! == +echo Your MySQL login information has been written to the file +echo "." +echo You must use the sql.mit.edu web interface to add or drop databases. +echo If you have trouble with your MySQL account, feel free to contact +echo the sql.mit.edu team by e-mailing sql@mit.edu +exit 0 Index: branches/fc20-dev/locker/bin/signup-web =================================================================== --- branches/fc20-dev/locker/bin/signup-web (revision 2523) +++ branches/fc20-dev/locker/bin/signup-web (revision 2523) @@ -0,0 +1,18 @@ +#!/bin/sh + +attach -q scripts +. "/mit/scripts/bin$scriptsdev/signup-minimal" + +if [ ! -d "/mit/$lname/web_scripts/" ]; then + mkdir -p "/mit/$lname/web_scripts" + fs sa "/mit/$lname/web_scripts" system:anyuser l + fs sa "/mit/$lname/web_scripts" system:authuser none + fs sa "/mit/$lname/web_scripts" system:scripts-security-upd rl + fs sa "/mit/$lname/web_scripts" daemon.scripts write + success "the web script service" "The directory /mit/$lname/web_scripts has been created." +else + success "the web script service" "The directory /mit/$lname/web_scripts already exists." +fi +if [ ! -f "/mit/$lname/web_scripts/.htaccess.mit" ]; then + ln -ns /afs/athena.mit.edu/contrib/scripts/www/web_scripts-.htaccess.mit /mit/$lname/web_scripts/.htaccess.mit +fi Index: branches/fc20-dev/locker/bin/ssh =================================================================== --- branches/fc20-dev/locker/bin/ssh (revision 2523) +++ branches/fc20-dev/locker/bin/ssh (revision 2523) @@ -0,0 +1,15 @@ +#!/bin/bash + +# add -r scripts +if [ -x /bin/athena/attach ]; then + eval $(/bin/athena/attach -Padd -b -r scripts) +fi + +exec ssh \ + -o GSSAPIAuthentication=yes \ + -o GSSAPIDelegateCredentials=no \ + -o PreferredAuthentications=gssapi-with-mic \ + -o ForwardX11=no \ + -o GlobalKnownHostsFile=/afs/athena.mit.edu/contrib/scripts/etc/known_hosts \ + -o UserKnownHostsFile=/dev/null \ + "$@" Index: branches/fc20-dev/locker/bin/sshmic =================================================================== --- branches/fc20-dev/locker/bin/sshmic (revision 2523) +++ branches/fc20-dev/locker/bin/sshmic (revision 2523) @@ -0,0 +1,1 @@ +link ssh Index: branches/fc20-dev/locker/bin/webaccess =================================================================== --- branches/fc20-dev/locker/bin/webaccess (revision 2523) +++ branches/fc20-dev/locker/bin/webaccess (revision 2523) @@ -0,0 +1,223 @@ +#!/bin/bash + +# webaccess +# Manage access control for scripts.mit.edu in .htaccess and .htpasswd files. +# Anders Kaseorg + +set -e + +on_exit= +trap 'eval "$on_exit"' EXIT + +dir="$(pwd)" +htaccess=$dir/.htaccess +authuserfile=$dir/.htpasswd + +add_users= +del_users= +enable_auth=1 +def_authname=\"Private\" + +begin_section="### BEGIN webaccess directives" +end_section="### END webaccess directives" + +usage () { + cat <&2 +usage: + webaccess -a Allow access from and set password. + webaccess -d Deny access from . + webaccess -r Reset default access control. +EOF + exit 1 +} + +getpass () { + user=$1 + ( + echo -n "New password for $user: " >/dev/tty + trap 'stty echo; echo >/dev/tty' EXIT + stty -echo + perl -e 'chop($_ = <>); print crypt($_, "\$1\$" . join "", (".", "/", "0".."9", "A".."Z", "a".."z") [rand 64, rand 64, rand 64, rand 64, rand 64, rand 64, rand 64, rand 64])' "$tmp_htaccess" + +config_written=0 +write_config () { + if [ $config_written -eq 1 ]; then return 0; fi + config_written=1 + if [ $enable_auth -eq 1 ]; then + echo "$begin_section" >&3 + echo "# See http://scripts.mit.edu/faq/23" >&3 + echo "AuthUserFile $authuserfile" >&3 + echo "AuthName ${authname:-$def_authname}" >&3 + echo "AuthType Basic" >&3 + echo "Require valid-user" >&3 + echo "$end_section" >&3 + fi +} + +if [ -e "$htaccess" ]; then + exec 4<"$htaccess" + + oldconfig_state=0 + oldconfig_buffer=__END__ + + while read -r line <&4; do + oldconfig_newstate=0 + case "$line" in + "AuthUserFile "*) oldconfig_newstate=1 ;; + "AuthName "*) oldconfig_newstate=2; oldconfig_authname=${line#AuthName } ;; + "AuthType Basic") oldconfig_newstate=3 ;; + "") oldconfig_newstate=4 ;; + "require valid-user") oldconfig_newstate=5 ;; + "") oldconfig_newstate=6 ;; + esac + + if [ $oldconfig_newstate -ne $(($oldconfig_state + 1)) ]; then + if [ $oldconfig_state -ne 0 ]; then + echo "${oldconfig_buffer% +__END__}" >&3 + oldconfig_state=0 + oldconfig_buffer=__END__ + fi + fi + + if [ "$line" = "$begin_section" ]; then + while read -r line <&4 && [ "$line" != "$end_section" ]; do + case "$line" in + "AuthName "*) + def_authname=${line#AuthName } + ;; + esac + done + write_config + elif [ $oldconfig_newstate -eq $(($oldconfig_state + 1)) ]; then + oldconfig_buffer=$(echo "${oldconfig_buffer%__END__}$line"; echo __END__) + oldconfig_state=$oldconfig_newstate + if [ $oldconfig_state -eq 6 ]; then + echo "Replacing obsolete webaccess configuration." >&2 + oldconfig_state=0 + oldconfig_buffer=__END__ + def_authname=$oldconfig_authname + fi + else + echo "$line" >&3 + fi + done + + if [ $oldconfig_state -ne 0 ]; then + echo "${oldconfig_buffer% +__END__}" + oldconfig_state=0 + oldconfig_buffer=__END__ + fi + + exec 4<&- +fi + +write_config + +exec 3>&- +if ! cmp -s "$htaccess" "$tmp_htaccess"; then + if [ -s "$tmp_htaccess" ]; then + echo "Updating $htaccess" >&2 + mv -f "$tmp_htaccess" "$htaccess" + else + if [ -e "$htaccess" ]; then + echo "Deleting $htaccess" >&2 + rm -f "$htaccess" + fi + rm -f "$tmp_htaccess" + fi +fi +trap - EXIT + +if [ $enable_auth -eq 1 ]; then + if [ ! -e "$authuserfile" ]; then touch "$authuserfile"; fi + + tmp_authuserfile=$authuserfile.webaccess-new + trap 'rm -f "$tmp_authuserfile"' EXIT + exec 3>"$tmp_authuserfile" + + exec 4<"$authuserfile" + while IFS=: read user pass <&4; do + for del_user in $del_users; do + if [ "$del_user" = "$user" ]; then + echo "Deleting user $del_user:" >&2 + pass= + fi + done + new_add_users= + for add_user in $add_users; do + if [ "$add_user" = "$user" ]; then + pass=$(getpass "$user") + else + new_add_users=$new_add_users\ $add_user + fi + done + add_users=$new_add_users + if [ -n "$pass" ]; then + echo "$user:$pass" >&3 + fi + done + exec 4<&- + + for add_user in $add_users; do + pass=$(getpass "$add_user") + echo "$add_user:$pass" >&3 + done + + exec 3>&- + mv -f "$tmp_authuserfile" "$authuserfile" + trap - EXIT + + echo "Done. New list of valid users:" >&2 + sed -n 's/^\([^:]*\):.*$/ \1/ p' "$authuserfile" +else + rm -f "$authuserfile" +fi Index: branches/fc20-dev/locker/cron/bin/cronload =================================================================== --- branches/fc20-dev/locker/cron/bin/cronload (revision 2523) +++ branches/fc20-dev/locker/cron/bin/cronload (revision 2523) @@ -0,0 +1,135 @@ +#!/usr/bin/perl + +# Author: + +use strict; +use warnings; + +use File::Spec::Functions; +use Getopt::Long; + +use constant { + CRON_DIR => "cron_scripts", + CRONTAB_FILE => "crontab", + AUTO_DIR => "AUTO", + SPOOL_DIR => "/mit/scripts/cron/crontabs", +}; + +my $force = 0; +my $list = 0; +my $pretend = 0; + +sub get_crontabs() { + my $crontab = catfile($ENV{"HOME"}, CRON_DIR, CRONTAB_FILE); + my $crontabdir = catdir($ENV{"HOME"}, CRON_DIR, AUTO_DIR); + + my @crontabs; + + opendir(CRONTABS, $crontabdir) or print "You don't have a ".CRON_DIR."/".AUTO_DIR."/ directory\n"; + push(@crontabs, grep { -r $_ } map { catfile($crontabdir, $_) } grep { !/^[\.#]/ } readdir(CRONTABS)); + closedir(CRONTABS); + + push (@crontabs, $crontab) if (-r $crontab); + return @crontabs; +} + +sub read_crontab($) { + my ($file) = @_; + # local $/; + + open(CRONTAB, $file) or die "Couldn't read crontab $file!"; + my @lines = ; + close(CRONTAB); + + return @lines; +} + +sub check_crontab(@) { + my (@lines) = @_; + + my @errors; + + foreach my $line (@lines) { + $line =~ s|#.*$||; # Remove comments + $line =~ s|^\s*(.*?)\s*$|$1|; # Remove whitespace + + if ($line =~ m|^\w[\w\d]*=|) { + # Comment + next; + } elsif ($line =~ m|^(?:(\S+)\s+){5}(.*)|) { + # Crontab line + my ($minute, $hour, $day, $month, $dow) = ($1,$2,$3,$4,$5); + # FIXME: Validate the time fields. + next; + } elsif ($line =~ m|^\s*$|) { + # Whitespace + next; + } else { + push(@errors, "Unrecognized crontab line:\n$line\n"); + } + } + return @errors; +} + + + +GetOptions("force|f+" => \$force, + "list|l" => \$list, + "pretend|p" => \$pretend); + +if ($list) { + my $file = catfile(SPOOL_DIR, $ENV{"USER"}); + local $/; + open (CRONTAB, $file) or die "No crontab installed.\n"; + print ; + close (CRONTAB); + exit; +} + +my @crontabs = get_crontabs(); +my @all_errors; +my @final_crontab; +my ($numvalid, $numinvalid) = (0,0); + +foreach my $crontab (@crontabs) { + push(@final_crontab, "### $crontab\n"); + my @crontab = read_crontab($crontab); + my @errors = check_crontab(@crontab); + if (@errors == 0) { + print "$crontab is a valid crontab\n"; + push(@final_crontab, @crontab); + $numvalid++; + } else { + print "$crontab has errors:\n"; + push(@all_errors, scalar(@errors)." errors in $crontab:\n", @errors); + print join("\n", @errors); + $numinvalid++; + if ($force >= 2) { + push(@final_crontab, @crontab); + } else { + my $errors = join("\n", @errors); + $errors =~ s|^|# |mg; + push(@final_crontab, "## $crontab was not installed due to errors:\n", $errors); + } + } +} +if ($pretend) { + print "Would install this crontab:\n"; + print @final_crontab; + exit; +} + +if ($force < 1 && @all_errors) { + print "Not loading new crontab. Use -f to force.\n"; + exit; +} +if ($force >= 2 && @all_errors) { + print "Loading $numvalid crontab ($numinvalid BROKEN!) files...\n"; +} else { + print "Loading $numvalid crontab files...\n"; +} + +# FIXME +# Load @final_crontab somehow + +print "done.\n"; Index: branches/fc20-dev/locker/cron/bin/crontab =================================================================== --- branches/fc20-dev/locker/cron/bin/crontab (revision 2523) +++ branches/fc20-dev/locker/cron/bin/crontab (revision 2523) @@ -0,0 +1,15 @@ +#!/bin/sh + +# Author: + +if [[ "$1" = "-l" ]]; then + `dirname $0`/cronload -l; +else + cat <pw_name); + + i = 1; + if (ac > 1) { + if (av[1][0] == '-' && av[1][1] == 0) { + option = REPLACE; + ++i; + } else if (av[1][0] != '-') { + option = REPLACE; + ++i; + repFile = av[1]; + } + } + + for (; i < ac; ++i) { + char *ptr = av[i]; + + if (*ptr != '-') + break; + ptr += 2; + + switch(ptr[-1]) { + case 'l': + if (ptr[-1] == 'l') + option = LIST; + /* fall through */ + case 'd': + if (ptr[-1] == 'd') + option = DELETE; + /* fall through */ + case 'u': + if (i + 1 < ac && av[i+1][0] != '-') { + ++i; + if (getuid() == geteuid()) { + pas = getpwnam(av[i]); + if (pas) { + UserId = pas->pw_uid; + } else { + errx(1, "user %s unknown\n", av[i]); + } + } else { + errx(1, "only the superuser may specify a user\n"); + } + } + break; + case 'c': + if ((getuid() == geteuid()) && (0 == getuid())) { + CDir = (*ptr) ? ptr : av[++i]; + } else { + errx(1, "-c option: superuser only\n"); + } + break; + default: + i = ac; + break; + } + } + if (i != ac || option == NONE) { + printf("cronload.real " VERSION "\n"); + printf("cronload.real file replace crontab from file\n"); + printf("cronload.real - replace crontab from stdin\n"); + printf("cronload.real -u user specify user\n"); + printf("cronload.real -l [user] list crontab for user\n"); + printf("cronload.real -d [user] delete crontab for user\n"); + printf("cronload.real -c dir specify crontab directory\n"); + exit(0); + } + + /* + * Get password entry + */ + + if ((pas = getpwuid(UserId)) == NULL) { + perror("getpwuid"); + exit(1); + } + + /* + * If there is a replacement file, obtain a secure descriptor to it. + */ + + if (repFile) { + repFd = GetReplaceStream(caller, repFile); + if (repFd < 0) { + errx(1, "unable to read replacement file\n"); + } + } + + /* + * Change directory to our crontab directory + */ + + if (chdir(CDir) < 0) { + errx(1, "cannot change dir to %s: %s\n", CDir, strerror(errno)); + } + + /* + * Handle options as appropriate + */ + + switch(option) { + case LIST: + { + FILE *fi; + char buf[1024]; + + if ((fi = fopen(pas->pw_name, "r"))) { + while (fgets(buf, sizeof(buf), fi) != NULL) + fputs(buf, stdout); + fclose(fi); + } else { + fprintf(stderr, "no crontab for %s\n", pas->pw_name); + } + } + break; + case REPLACE: + { + char buf[1024]; + char path[1024]; + int fd; + int n; + + snprintf(path, sizeof(path), "%s.new", pas->pw_name); + if ((fd = open(path, O_CREAT|O_TRUNC|O_EXCL|O_APPEND|O_WRONLY, 0600)) >= 0) { + while ((n = read(repFd, buf, sizeof(buf))) > 0) { + write(fd, buf, n); + } + close(fd); + rename(path, pas->pw_name); + } else { + fprintf(stderr, "unable to create %s/%s: %s\n", + CDir, + path, + strerror(errno) + ); + } + close(repFd); + } + break; + case DELETE: + remove(pas->pw_name); + break; + case NONE: + default: + break; + } + + /* + * Bump notification file. Handle window where crond picks file up + * before we can write our entry out. + */ + /* // only applicable to dcron + if (option == REPLACE || option == DELETE) { + FILE *fo; + struct stat st; + + while ((fo = fopen(CRONUPDATE, "a"))) { + fprintf(fo, "%s\n", pas->pw_name); + fflush(fo); + if (fstat(fileno(fo), &st) != 0 || st.st_nlink != 0) { + fclose(fo); + break; + } + fclose(fo); + // * loop * / + } + if (fo == NULL) { + fprintf(stderr, "unable to append to %s/%s\n", CDir, CRONUPDATE); + } + } + */ + (volatile void)exit(0); + /* not reached */ +} + +int +GetReplaceStream(const char *user, const char *file) +{ + int filedes[2]; + int pid; + int fd; + int n; + char buf[1024]; + + if (pipe(filedes) < 0) { + perror("pipe"); + return(-1); + } + if ((pid = fork()) < 0) { + perror("fork"); + return(-1); + } + if (pid > 0) { + /* + * PARENT + */ + + close(filedes[1]); + if (read(filedes[0], buf, 1) != 1) { + close(filedes[0]); + filedes[0] = -1; + } + return(filedes[0]); + } + + /* + * CHILD + */ + + close(filedes[0]); + + if (ChangeUser(user, 0) < 0) + exit(0); + + fd = open(file, O_RDONLY); + if (fd < 0) + errx(0, "unable to open %s\n", file); + buf[0] = 0; + write(filedes[1], buf, 1); + while ((n = read(fd, buf, sizeof(buf))) > 0) { + write(filedes[1], buf, n); + } + exit(0); +} Index: branches/fc20-dev/locker/cron/src/defs.h =================================================================== --- branches/fc20-dev/locker/cron/src/defs.h (revision 2523) +++ branches/fc20-dev/locker/cron/src/defs.h (revision 2523) @@ -0,0 +1,46 @@ + +/* + * DEFS.H + * + * Copyright 1994-1998 Matthew Dillon (dillon@backplane.com) + * May be distributed under the GNU General Public License + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define Prototype extern +#define arysize(ary) (sizeof(ary)/sizeof((ary)[0])) + +#ifndef SCRIPTS_CRONTABS +#define SCRIPTS_CRONTABS "/mit/scripts/cron/crontabs" +#endif +#ifndef TMPDIR +#define TMPDIR "/tmp" +#endif +#ifndef OPEN_MAX +#define OPEN_MAX 256 +#endif + +#ifndef CRONUPDATE +#define CRONUPDATE "cron.update" +#endif + +#ifndef MAXLINES +#define MAXLINES 256 /* max lines in non-root crontabs */ +#endif Index: branches/fc20-dev/locker/cron/src/subs.c =================================================================== --- branches/fc20-dev/locker/cron/src/subs.c (revision 2523) +++ branches/fc20-dev/locker/cron/src/subs.c (revision 2523) @@ -0,0 +1,147 @@ + +/* + * SUBS.C + * + * Copyright 1994 Matthew Dillon (dillon@apollo.backplane.com) + * May be distributed under the GNU General Public License + */ + +#include "defs.h" + +Prototype void logn(int level, const char *ctl, ...); +Prototype void log9(const char *ctl, ...); +Prototype void logfd(int fd, const char *ctl, ...); +Prototype void fdprintf(int fd, const char *ctl, ...); +Prototype int ChangeUser(const char *user, short dochdir); +Prototype void vlog(int level, int fd, const char *ctl, va_list va); +Prototype int slog(char *buf, const char *ctl, int nmax, va_list va, short useDate); + +extern short LogLevel; + +void +log9(const char *ctl, ...) +{ + va_list va; + + va_start(va, ctl); + vlog(9, 2, ctl, va); + va_end(va); +} + +void +logn(int level, const char *ctl, ...) +{ + va_list va; + + va_start(va, ctl); + vlog(level, 2, ctl, va); + va_end(va); +} + +void +logfd(int fd, const char *ctl, ...) +{ + va_list va; + + va_start(va, ctl); + vlog(9, fd, ctl, va); + va_end(va); +} + +void +fdprintf(int fd, const char *ctl, ...) +{ + va_list va; + char buf[2048]; + + va_start(va, ctl); + vsnprintf(buf, sizeof(buf), ctl, va); + write(fd, buf, strlen(buf)); + va_end(va); +} + +void +vlog(int level, int fd, const char *ctl, va_list va) +{ + char buf[2048]; + short n; + static short useDate = 1; + + if (level >= LogLevel) { + write(fd, buf, n = slog(buf, ctl, sizeof(buf), va, useDate)); + useDate = (n && buf[n-1] == '\n'); + } +} + +int +slog(char *buf, const char *ctl, int nmax, va_list va, short useDate) +{ + time_t t = time(NULL); + struct tm *tp = localtime(&t); + + buf[0] = 0; + if (useDate) + strftime(buf, 128, "%d-%b-%y %H:%M ", tp); + vsnprintf(buf + strlen(buf), nmax, ctl, va); + return(strlen(buf)); +} + +int +ChangeUser(const char *user, short dochdir) +{ + struct passwd *pas; + + /* + * Obtain password entry and change privilages + */ + + if ((pas = getpwnam(user)) == 0) { + logn(9, "failed to get uid for %s", user); + return(-1); + } + setenv("USER", pas->pw_name, 1); + setenv("HOME", pas->pw_dir, 1); + setenv("SHELL", "/bin/sh", 1); + + /* + * Change running state to the user in question + */ + + if (initgroups(user, pas->pw_gid) < 0) { + logn(9, "initgroups failed: %s %s", user, strerror(errno)); + return(-1); + } + if (setregid(pas->pw_gid, pas->pw_gid) < 0) { + logn(9, "setregid failed: %s %d", user, pas->pw_gid); + return(-1); + } + if (setreuid(pas->pw_uid, pas->pw_uid) < 0) { + logn(9, "setreuid failed: %s %d", user, pas->pw_uid); + return(-1); + } + if (dochdir) { + if (chdir(pas->pw_dir) < 0) { + logn(8, "chdir failed: %s %s", user, pas->pw_dir); + if (chdir(TMPDIR) < 0) { + logn(9, "chdir failed: %s %s", user, pas->pw_dir); + logn(9, "chdir failed: %s " TMPDIR, user); + return(-1); + } + } + } + return(pas->pw_uid); +} + +#if 0 + +char * +strdup(const char *str) +{ + char *ptr = malloc(strlen(str) + 1); + + if (ptr) + strcpy(ptr, str); + return(ptr); +} + +#endif Index: branches/fc20-dev/locker/deploy/bin/django =================================================================== --- branches/fc20-dev/locker/deploy/bin/django (revision 2523) +++ branches/fc20-dev/locker/deploy/bin/django (revision 2523) @@ -0,0 +1,134 @@ +#!/usr/bin/perl +use strict; +use FindBin qw($Bin); +use lib $Bin; +use onserver; + +setup(); + +print "\nEnter the code name for your project (a valid Python package name).\n"; +print "Do not use 'django' or the name of any other Python library.\n"; +print "Project name: "; +my $name = ; +chomp $name; + +open FASTCGI, ">index.fcgi"; +print FASTCGI <.htaccess"; +print HTACCESS <settings.py.new"; +while () { + chomp; + if (/Your Name/) { + $_ = " ('$USER', '$email'),"; + } elsif (/^DEBUG = /) { + $_ =~ s/DEBUG/import os\n\nDEBUG/; + } elsif (/'ENGINE'/) { + $_ = " 'ENGINE': 'django.db.backends.mysql',"; + } elsif (/'NAME'/) { + $_ = " 'NAME': '$sqldb',"; + } elsif (/'USER'/) { + $_ = " 'OPTIONS': {\n 'read_default_file' : os.path.expanduser('~/.my.cnf'),\n },"; + } elsif (/'PASSWORD'/) { + next; + } elsif (/'HOST'/) { + next; + } elsif (/Chicago/) { + $_ =~ s/Chicago/New_York/; + } elsif (/^ADMIN_MEDIA_PREFIX/) { + $_ = "ADMIN_MEDIA_PREFIX = '/__scripts/django/media/'"; + } elsif (/^INSTALLED_APPS/) { + print NEWSETTINGS "$_\n"; + while () { + if (/^\)/) { + print NEWSETTINGS " 'django.contrib.admin',\n"; + print NEWSETTINGS " 'django.contrib.admindocs',\n"; + } + print NEWSETTINGS $_; + } + } + print NEWSETTINGS "$_\n"; +} +close NEWSETTINGS; +close SETTNGS; +rename "settings.py.new", "settings.py"; + +open URLS, "urls.py"; +open NEWURLS, ">urls.py.new"; +while () { + chomp; + if (/^#.*from django\.contrib import admin/) { + $_ =~ s/^# *//; + } elsif (/^#.*admin.autodiscover/) { + $_ =~ s/^# *//; + } elsif (/^ *# *\(r\'\^admin\//) { + $_ =~ s/# *//; + } + print NEWURLS "$_\n"; +} +close NEWURLS; +close URLS; +rename "urls.py.new", "urls.py"; + +chdir ".."; + +print "Initializing your project's SQL database schema...\n"; +system qw{./manage.py syncdb --noinput}; +print "...done\n"; + +print "Creating your superuser account... "; +system qw{./manage.py createsuperuser --username}, $admin_username, "--email", $email, "--noinput"; +print "done\n"; +print "Setting your superuser password... "; +system qw{mysql -D}, "$USER+$addrlast", "-e", "UPDATE auth_user SET password=MD5(\'$admin_password\') WHERE username=\'$admin_username\'"; +print "done\n"; + +print "\nDjango has been installed. The setup is roughly what's described\n"; +print "in the shared-hosting section of\n"; +print " http://docs.djangoproject.com/en/dev/howto/deployment/fastcgi/\n"; +print "We've also enabled the admin app. You can start from the 'Creating\n"; +print "models' step of the Django tutorial:\n"; +print " http://docs.djangoproject.com/en/dev/intro/tutorial01/#id3\n\n"; +print "Your project is located in:\n"; +print " /mit/$USER/Scripts/django/$name/\n"; +print "To access manage.py, run 'ssh -k $USER\@scripts' and cd to the above directory.\n\n"; +press_enter; + +exit 0; Index: branches/fc20-dev/locker/deploy/bin/gallery2 =================================================================== --- branches/fc20-dev/locker/deploy/bin/gallery2 (revision 2523) +++ branches/fc20-dev/locker/deploy/bin/gallery2 (revision 2523) @@ -0,0 +1,62 @@ +#!/usr/bin/perl +use strict; +use FindBin qw($Bin); +use lib $Bin; +use onserver_star; +use File::Path + +setup(); + +my $dir = "/mit/$USER/scripts-gallery2/$sqldb"; +if (-e $dir) { + print STDERR "ERROR: The directory $dir already exists.\nTry selecting a different name."; + exit 1; +} +mkpath($dir); + +system("patch", "install/steps/AuthenticateStep.class", + "/mit/scripts/deploy$scriptsdev/gallery2.patch"); + +$ua->cookie_jar({file => '.cookies'}); + +fetch_uri('install/index.php'); +fetch_uri('install/index.php', {step => 1}, + {language => 'en_US'}); +fetch_uri('install/index.php', {step => 2}, {}); +fetch_uri('install/index.php', {step => 3}, {}); + +my $post4 = {isMultisite => 0, + dir => $dir, + action => 'save'}; +fetch_uri('install/index.php', {step => 4}, $post4); +fetch_uri('install/index.php', {step => 4}, $post4); + +my $post5 = {type => 'mysql', + hostname => $sqlhost, + action => 'save', + confirmReuseTables => '', + confirmCleanInstall => '', + username => $sqluser, + password => $sqlpass, + database => $sqldb, + tablePrefix => 'g2_', + columnPrefix => 'g_'}; +fetch_uri('install/index.php', {step => 5}, $post5); +fetch_uri('install/index.php', {step => 5}, $post5); + +fetch_uri('install/index.php', {step => 6}, + {adminName => $admin_username, + passwordA => $admin_password, + action => 'create', + passwordB => $admin_password, + email => $email, + fullName => $USER}); +fetch_uri('install/index.php', {step => 7}, {}); +fetch_uri('install/index.php', {step => 8}, {}); +fetch_uri('install/index.php', {step => 9}, + {'module[imagemagick]' => 'on', + activate => 1}); +fetch_uri('install/index.php', {step => 10}, {}); +fetch_uri('install/index.php', {step => 11}, {}); + +unlink '.cookies'; Index: branches/fc20-dev/locker/deploy/bin/git =================================================================== --- branches/fc20-dev/locker/deploy/bin/git (revision 2523) +++ branches/fc20-dev/locker/deploy/bin/git (revision 2523) @@ -0,0 +1,80 @@ +#!/usr/bin/perl +use strict; +use FindBin qw($Bin); +use lib $Bin; +use onserver; + +setup(); + +my $gitbase = "$scriptsdir/git"; +my $htpasswd = "$gitbase/$addrend.git/.htpasswd"; + +open HTACCESS, ">.htaccess"; +print HTACCESS < + AuthName "Git Access" + AuthType basic + AuthUserFile $htpasswd + Require user $admin_username + + # Alternatively, replace "require user" with: + #Require group somegroup + #AuthGroupFile $gitbase/$addrend/.htgroup + # and set up .htgroup appropriately + + +RewriteRule ^($addrend\\.git/.*)\$ /~$USER/$addrend/_git.cgi/\$1 +EOF +close HTACCESS; +chmod 0777, ".htaccess"; + +open GIT_CGI, ">_git.cgi"; +print GIT_CGI <&2 + exit 1;; + /$addrend.git/*) + # pass + ;; + *) + echo "Content-type: text/plain" + echo "Status: 403 Forbidden" + echo "" + echo "Error: PATH_INFO='\$PATH_INFO' must start with /$addrend.git/" + echo "gitautoinstaller: \$HOME: found bad start in PATH_INFO='\$PATH_INFO'" >&2 + exit 1;; +esac +export GIT_PROJECT_ROOT="$gitbase" +export PATH_TRANSLATED="\$GIT_PROJECT_ROOT\$PATH_INFO" +export GIT_HTTP_EXPORT_ALL=1 +exec git http-backend +EOF +close GIT_CGI; +chmod 0755, "_git.cgi"; +symlink "_git.cgi","_git-auth.cgi"; + +chdir $gitbase; +system qw{git init --bare}, "$addrend.git"; +chdir "$addrend.git"; + +system qw{htpasswd -c}, $htpasswd, $admin_username; + +print "Your git repository is located in:\n"; +print " $gitbase/$addrend.git/\n"; +print "To clone, run\n git clone https://$USER.scripts.mit.edu/$addrend/$addrend.git\n\n"; +print "Note: Push over HTTP is a relatively new feature in Git, so if git push fails\n"; +print "try a newer version of Git, e.g. if you're on Athena, 'add -f git' and try again.\n\n"; +press_enter; + +exit 0; Index: branches/fc20-dev/locker/deploy/bin/joomla =================================================================== --- branches/fc20-dev/locker/deploy/bin/joomla (revision 2523) +++ branches/fc20-dev/locker/deploy/bin/joomla (revision 2523) @@ -0,0 +1,86 @@ +#!/usr/bin/perl +use strict; +use FindBin qw($Bin); +use lib $Bin; +use onserver; +use File::Path; + +setup(); + +print "\nPlease decide upon a title for your site and enter it below.\n"; +print "Desired title: "; +my $title = ; +chomp($title); + +print "\nSetting up your configuration file...\n"; +open CONFIGTEMPLATE, "configuration.php-dist"; +open CONFIGURATION, ">configuration.php"; +while () { + chomp; + if (/var \$sitename /) { + $_ = " var \$sitename = '$title';"; + } elsif (/var \$dbtype /) { + $_ = " var \$dbtype = 'mysql';"; + } elsif (/var \$host /) { + $_ = " var \$host = '$sqlhost';"; + } elsif (/var \$user /) { + $_ = " var \$user = '$sqluser';"; + } elsif (/var \$password /) { + $_ = " var \$password = '$sqlpass';"; + } elsif (/var \$db /) { + $_ = " var \$db = '$sqldb';"; + } elsif (/var \$dbprefix /) { + $_ = " var \$dbprefix = 'jos_';"; + } elsif (/var \$secret /) { + my $random = `dd if=/dev/urandom bs=1k count=1 | md5sum | cut -c1-32`; + $random =~ s/\n//; + $_ = " var \$secret = '$random';"; + } elsif (/var \$mailfrom /) { + $_ = " var \$mailfrom = '$email';"; + } elsif (/var \$fromname /) { + $_ = " var \$fromname = '$title';"; + } + print CONFIGURATION "$_\n"; +} +close CONFIGURATION; +close CONFIGTEMPLATE; + +print "\nInitializing database schema...\n"; +my $schemafile = "installation/sql/mysql/joomla-real.sql"; +open SCHEMATEMPLATE, "installation/sql/mysql/joomla.sql"; +open SCHEMA, ">$schemafile"; +while () { + if (/#__/) { + $_ =~ s/#__/jos_/g; + } + print SCHEMA "$_"; +} +close SCHEMA; +close SCHEMATEMPLATE; +system("cat $schemafile | mysql $sqldb"); + +print "\nLoading sample data...\n"; +my $sampledatafile = "installation/sql/mysql/sample_data-real.sql"; +open SAMPLETEMPLATE, "installation/sql/mysql/sample_data.sql"; +open SAMPLE, ">$sampledatafile"; +while () { + if (/#__/) { + $_ =~ s/#__/jos_/g; + } + print SAMPLE "$_"; +} +close SAMPLE; +close SAMPLETEMPLATE; +system("cat $sampledatafile | mysql $sqldb"); + +print "\nCreating your admin account...\n"; +system("mysql -e \"INSERT INTO jos_users VALUES (62, 'Administrator', '$admin_username', '$email', MD5('$admin_password'), 'Super Administrator', 0, 1, 25, NOW(), NOW(), '', '')\" $sqldb"); +system("mysql -e \"INSERT INTO jos_core_acl_aro VALUES (10, 'users', '62', 0, 'Administrator', 0)\" $sqldb"); +system("mysql -e \"INSERT INTO jos_core_acl_groups_aro_map VALUES (25, '', 10)\" $sqldb"); + +print "\nCleaning up installation tree...\n"; +rmtree('installation'); + +print "\nDone!\n"; +exit 0; + Index: branches/fc20-dev/locker/deploy/bin/mediawiki =================================================================== --- branches/fc20-dev/locker/deploy/bin/mediawiki (revision 2523) +++ branches/fc20-dev/locker/deploy/bin/mediawiki (revision 2523) @@ -0,0 +1,36 @@ +#!/usr/bin/perl +use strict; +use FindBin qw($Bin); +use lib $Bin; +use onserver; + +setup(); + +`cp skins/common/images/mediawiki.png skins/common/images/wiki.png`; +`patch -s -p1 < /mit/scripts/deploy/mediawiki.patch`; + +print "\nPlease decide upon a title for your wiki and enter it below.\n"; +print "Desired title: "; +my $title=; +chomp($title); + +my $html = fetch_uri( + 'config/index.php', + {}, + {Sitename => $title, + EmergencyContact => $email, + LanguageCode => 'en', + DBserver => $sqlhost, + DBname => $sqldb, + DBuser => $sqluser, + DBpassword => $sqlpass, + DBpassword2 => $sqlpass, + defaultEmail => $email, + SysopName => $admin_username, + SysopPass => $admin_password, + SysopPass2 => $admin_password}); +unless ($html =~ /Installation successful/) { + print STDERR "ERROR: Automatic MediaWiki configuration failed. You will need to configure\nyour MediaWiki manually, or email scripts\@mit.edu for help.\n"; + exit; +} +`cp config/LocalSettings.php .`; Index: branches/fc20-dev/locker/deploy/bin/onathena =================================================================== --- branches/fc20-dev/locker/deploy/bin/onathena (revision 2523) +++ branches/fc20-dev/locker/deploy/bin/onathena (revision 2523) @@ -0,0 +1,290 @@ +#!/bin/sh + +: ${aicontact:=scripts@mit.edu} +: ${aimaintainer:=scripts.mit.edu} +: ${ailocker:=scripts} + + +checkfailed() { + if [ -f "$lroot/web_scripts/$addrend/.failed" ]; then + rm -f "$lroot/web_scripts/$addrend/.failed" + exit 1 + fi +} + +die() { + echo "== INSTALLATION FAILED ==" + echo "Sorry, the installation failed:" + echo "$@" + echo "Please contact $aicontact and provide a copy of the output of this installer." + exit 1 +} + +attach scripts +[ "$ailocker" != "scripts" ] && attach "$ailocker" + +sshrun() { + /afs/athena.mit.edu/contrib/scripts/bin$scriptsdev/scripts-ssh "$lname" "$@" 2>/dev/null +} + +vsshrun() { + /afs/athena.mit.edu/contrib/scripts/bin$scriptsdev/scripts-ssh "$lname" "$@" +} + +checksqlpass() { + errors=`sshrun "/mit/scripts/sql/bin$scriptsdev/test-password"` + if [ "$errors" != "" ]; then + if [ "$1" -eq 1 ]; then + rm -f "$lroot/.sql/my.cnf" + fi + echo + echo "ERROR:" + printf "$2" + exit + fi +} + +override=1 +if [ "$override" = "" ]; then +echo "The $aimaintainer automatic installers are currently unavailable." +echo "We hope to make them available again soon." +echo "If you would like us to notify you as soon as they are available again," +echo "let us know by sending us an e-mail at $aicontact" +exit +fi + +echo +echo "== Welcome to the $aimaintainer installer for $sname ==" +echo + +echo "For documentation, including a link to the Athena rules of use," +echo "see ." +echo +echo "Please report problems with this installer to $aicontact." + +if [ -n "$SCRIPTS_INSTALL_LOCKER" ]; then + lname=$SCRIPTS_INSTALL_LOCKER +else + echo + echo "Are you performing this install for:" + echo "1. Your personal Athena account" + echo "2. A locker that you control (a club, a course, etc)" + echo "If you do not understand this question, you should answer '1'." + printf "Please enter either '1' or '2' (without quotes): " + read whofor + if [ "$whofor" = 1 ]; then + lname="${ATHENA_USER:-$USER}" + elif [ "$whofor" = 2 ]; then + echo + echo "OK. $sname will be installed into a locker of your choice that" + echo "you control. Please enter the name of the selected locker below." + echo "(For the locker /mit/lsc -- which has a full path of" + echo "/afs/athena.mit.edu/activity/l/lsc -- you would simply enter lsc)." + printf "Locker name: " + read lname + else + echo + echo "ERROR:" + echo "You must select either '1' or '2'." + exit 1 + fi + while true; do + if attach "$lname"; then + break + fi + echo "$lname is not a valid locker name." + printf "Locker name: " + read lname + done + unset whofor +fi +lroot="/mit/$lname" + +echo +echo Checking the status of your scripts.mit.edu account... + +attach scripts 2>/dev/null +. "/mit/scripts/bin$scriptsdev/signup-web" + +if [ -n "$SCRIPTS_INSTALL_ADDREND" ]; then + addrend=$SCRIPTS_INSTALL_ADDREND +else + echo + echo "Your new copy of $sname will appear on the web at a URL" + echo "that starts with http://$lname.scripts.mit.edu/" + echo "Please decide upon a complete URL and enter it below." + echo "You must enter one or more characters after mit.edu/" + echo "The completed address must only contain a-z, 0-9, and /." + printf "Desired address: http://$lname.scripts.mit.edu/" + read addrend +fi + +addrend=`perl -0e 'print $ARGV[0] =~ /^([\w\/-]*[\w-])\/*$/' -- "$addrend"` +if [ "$addrend" = "" ]; then + echo + echo "ERROR:" + echo "You must enter one or more characters after mit.edu/" + echo "The completed address must only contain a-z, 0-9, and /." + exit 1 +fi + +if [ -d "$lroot/web_scripts/$addrend" ]; then + echo + echo "ERROR:" + echo "You already have a directory corresponding to that web address." + echo "Please remove that directory, choose a different address, or" + echo "contact $aicontact for assistance." + exit 1 +fi + +if [ "$requires_sql" = "" ]; then + requires_sql=1 +fi + +if [ ! -f "$lroot/.my.cnf" ]; then + mkdir "$lroot/.sql" 2>/dev/null + fs sa "$lroot/.sql" daemon.scripts write + fs sa "$lroot/.sql" daemon.sql write + ln -nfs "$lroot/.sql/my.cnf" "$lroot/.my.cnf" 2>/dev/null +fi + +fs sa "$lroot/.sql" system:anyuser none +fs sa "$lroot/.sql" system:authuser none + +if [ "$requires_sql" -eq 1 ]; then + sqlinfo=`sshrun "/mit/scripts/sql/bin$scriptsdev/get-password"` + if [ "$sqlinfo" = "" ]; then + echo + echo "You already have a MySQL account but you do not have a .my.cnf file." + echo "If you do not remember your MySQL account password, you can change it" + echo "at http://sql.mit.edu using MIT certificates." + printf "Please type your MySQL password and press [enter]: " + stty -echo + read sqlpass + stty echo + echo + sqlhost="sql.mit.edu" + sqluser=$lname + . "/mit/scripts/sql/bin$scriptsdev/save-password" + checksqlpass 1 'The MySQL password that you typed appears to be incorrect.\n' + echo + echo "OK. Continuing with the install..." + else + checksqlpass 0 'The MySQL login information in your .my.cnf file\nappears to be incorrect.\n' + fi +fi + +origdir=`pwd` +mkdir -p "$lroot/web_scripts_tmp" +cd "$lroot/web_scripts_tmp" +fs sa . system:anyuser none +fs sa . system:authuser none +fs sa . daemon.scripts write +fs sa . system:scripts-security-upd write +echo "This directory is necessary to store login sessions and other transient files for auto-installed packages from scripts.mit.edu." > DO_NOT_DELETE.txt +mkdir -p "$lroot/web_scripts/$addrend" +cd "$lroot/web_scripts/$addrend" +fs sa . system:anyuser none +fs sa . system:authuser none +fs sa . daemon.scripts write +fs sa . system:scripts-security-upd write + +# This version is deprecated, use create_scripts_dir instead + +if [ "$create_dir" = "" ]; then + create_dir=0 +fi + +if [ "$create_dir" -eq 1 ]; then + mkdir -p "$lroot/scripts-$deploy" + fs sa "$lroot/scripts-$deploy" system:anyuser none + fs sa "$lroot/scripts-$deploy" system:authuser none + fs sa "$lroot/scripts-$deploy" daemon.scripts write + fs sa "$lroot/scripts-$deploy" system:scripts-security-upd write +fi + +# This is the better version + +if [ "$create_scripts_dir" = "" ]; then + create_scripts_dir=0 +fi + +if [ "$create_scripts_dir" -eq 1 ]; then + mkdir -p "$lroot/Scripts/$deploy" + fs sa "$lroot/Scripts/$deploy" system:anyuser none + fs sa "$lroot/Scripts/$deploy" system:authuser none + fs sa "$lroot/Scripts/$deploy" daemon.scripts write + fs sa "$lroot/Scripts/$deploy" system:scripts-security-upd write +fi + +if [ "$wizard" != "" ]; then + if [ "$create_scripts_dir" -eq 1 ]; then + vsshrun "/mit/$ailocker/wizard/bin/wizard" "install" "--web-stub-path" "$lroot/web_scripts/$addrend" "$@" "$wizard" "$lroot/Scripts/$deploy/$addrend" + else + vsshrun "/mit/$ailocker/wizard/bin/wizard" "install" "$@" "$wizard" "$lroot/web_scripts/$addrend" + fi + exit 0 +fi + +if [ "$prompt_username" = "" ]; then + admin_username="admin" + prompt_username=0 +fi +if [ "$prompt_password" = "" ]; then + prompt_password=1 +fi + +if [ "$prompt_username" -eq 1 ]; then + echo + echo "You will be able to log in to $sname using a username of your choice." + echo "Please decide upon a username and enter it below." + echo "Your username must contain only alphanumeric characters (a-z, 0-9)." + printf "Desired username: " + read admin_username + admin_username=`perl -0e 'print $ARGV[0] =~ /^([[:alnum:]]+)$/' -- "$admin_username"` + if [ "$admin_username" = "" ]; then + echo + echo ERROR: + echo "Your username must contain only alphanumeric characters (a-z, 0-9)." + echo "You will need to run the installer again and choose a different username." + exit 1 + fi +fi + +if [ "$prompt_password" -eq 1 ]; then + stty -echo + sshrun "/mit/$ailocker/deploy$scriptsdev/bin/prompt-password" "$sname" "$deploy" "$addrend" "$admin_username" + stty echo +fi + +echo +echo "Unpacking $sname... (this step might take several minutes)" +# xavid: use p to keep the same permissions as in the file +athrun scripts gtar zxpf "/mit/$ailocker/deploy$scriptsdev/$deploy.tar.gz" +files=`athrun scripts gfind . -mindepth 1 -maxdepth 1 | grep -v .admin` +numfiles=`echo "$files" | wc -l` +if [ ! -z "$files" ]; then + if [ "$numfiles" -eq 1 ]; then + athrun scripts gfind . -mindepth 2 -maxdepth 2 | xargs -i mv \{} . + rmdir "$files" + fi +fi +if [ -f "/mit/$ailocker/deploy$scriptsdev/php.ini/$deploy" ]; then + nodot=`echo "$lname" | sed "/\./s///"`; + sed -e "/SCRIPTS_USER/ s//$lname/" -e "/SCRIPTS_NODOT/ s//$nodot/" "/mit/$ailocker/deploy$scriptsdev/php.ini/$deploy" > php.ini + athrun scripts gfind . -mindepth 1 -type d -exec sh -c 'ln -sf "`echo "$1" | sed '\''s,[^/],,g; s,/,../,g'\''`php.ini" "$1/"' -- {} \; +fi +cd "$origdir" + +vsshrun "/mit/$ailocker/deploy$scriptsdev/bin/$deploy" "$sname" "$deploy" "$addrend" "$admin_username" "$requires_sql" "$scriptsdev" "${ATHENA_USER:-$USER}" || die "Unknown failure during configuration" +rm -f "$lroot/web_scripts/$addrend/.scripts-tmp" +checkfailed + +echo +echo "== Installation complete! ==" +echo "You should now be able to access your new copy of $sname at" +echo "http://$lname.scripts.mit.edu/$addrend/" +echo "(You can replace the http with https if you want to use encryption)" +echo "If you have trouble accessing it, feel free to contact" +echo "the $aimaintainer team by e-mailing $aicontact" +exit 0 Index: branches/fc20-dev/locker/deploy/bin/onserver.pm =================================================================== --- branches/fc20-dev/locker/deploy/bin/onserver.pm (revision 2523) +++ branches/fc20-dev/locker/deploy/bin/onserver.pm (revision 2523) @@ -0,0 +1,145 @@ +package onserver; +use strict; +use Exporter; +use Sys::Hostname; +use File::Spec::Functions; +use File::Basename; +use Socket; +use Cwd qw(abs_path); +use POSIX qw(strftime); +use LWP::UserAgent; +use URI; +our @ISA = qw(Exporter); +our @EXPORT = qw(setup totmp fetch_uri print_login_info press_enter $server $tmp $USER $HOME $scriptsdir $sname $deploy $addrend $base_uri $ua $admin_username $requires_sql $addrlast $sqlhost $sqluser $sqlpass $sqldb $admin_password $scriptsdev $human $email); + +our $server = "scripts.mit.edu"; + +our ($tmp, $USER, $HOME, $scriptsdir, $sname, $deploy, $addrend, $base_uri, $ua, $admin_username, $requires_sql, $addrlast, $sqlhost, $sqluser, $sqlpass, $sqldb, $admin_password, $scriptsdev, $human, $email); + +$tmp = ".scripts-tmp"; +sub totmp { + open(FILE, ">$tmp"); + print FILE $_[0]; + close(FILE); +} + +$ua = LWP::UserAgent->new; +push @{$ua->requests_redirectable}, 'POST'; + +sub fetch_uri { + my ($uri, $get, $post) = @_; + my $u = URI->new($uri); + my $req; + if (defined $post) { + $u->query_form($post); + my $content = $u->query; + $u->query_form($get); + $req = HTTP::Request->new(POST => $u->abs($base_uri)); + $req->content_type('application/x-www-form-urlencoded'); + $req->content($content); + } else { + $u->query_form($get) if (defined $get); + $req = HTTP::Request->new(GET => $u->abs($base_uri)); + } + my $res = $ua->request($req); + if ($res->is_success) { + return $res->content; + } else { + print STDERR "Error fetching configuration page: ", $res->status_line, "\n"; + return undef; + } +} + +sub print_login_info { + print "\nYou will be able to log in to $sname using the following:\n"; + print " username: $admin_username\n"; + print " password: $admin_password\n"; +} + +sub getclienthostname { + if (my $sshclient = $ENV{"SSH_CLIENT"}) { + my ($clientip) = split(' ', $sshclient); + my $hostname = gethostbyaddr(inet_aton($clientip), AF_INET); + return $hostname || $clientip; + } else { + return hostname(); + } +} + +sub press_enter { + local $/ = "\n"; + print "Press [enter] to continue with the install."; + my $enter = ; +} + +sub setup { + $ENV{PATH} = '/bin:/usr/bin'; + $USER = $ENV{USER}; + $HOME = $ENV{HOME}; + $scriptsdir = $HOME; + $scriptsdir =~ s/\/Scripts$//; + $scriptsdir .= "/Scripts"; + + ($sname, $deploy, $addrend, $admin_username, $requires_sql, $scriptsdev, $human) = @ARGV; + chdir "$HOME/web_scripts/$addrend"; + $email = "$human\@mit.edu"; + + if($addrend =~ /^(.*)\/$/) { + $addrend = $1; + } + ($addrlast) = ($addrend =~ /([^\/]*)$/); + + $base_uri = "http://$server/~$USER/$addrend/"; + + if($requires_sql) { + print "\nCreating SQL database for $sname...\n"; + + open GETPWD, '-|', "/mit/scripts/sql/bin$scriptsdev/get-password"; + ($sqlhost, $sqluser, $sqlpass) = split(/\s/, ); + close GETPWD; + open SQLDB, '-|', "/mit/scripts/sql/bin$scriptsdev/get-next-database", $addrlast; + $sqldb = ; + close SQLDB; + open SQLDB, '-|', "/mit/scripts/sql/bin$scriptsdev/create-database", $sqldb; + $sqldb = ; + close SQLDB; + if($sqldb eq "") { + print "\nERROR:\n"; + print "Your SQL account failed to create a SQL database.\n"; + print "You should log in at http://sql.mit.edu to check whether\n"; + print "your SQL account is at its database limit or its storage limit.\n"; + print "If you cannot determine the cause of the problem, please\n"; + print "feel free to contact sql\@mit.edu for assistance.\n"; + open FAILED, ">.failed"; + close FAILED; + exit 1; + } + } + + if(-e "$HOME/web_scripts/$addrend/.admin") { + open ADMIN, "<$HOME/web_scripts/$addrend/.admin"; + $admin_password=; + chomp($admin_password); + close ADMIN; + unlink "$HOME/web_scripts/$addrend/.admin"; + } + + print "\nConfiguring $sname...\n"; + if($requires_sql) { + print "A copy of ${USER}'s SQL login info will be placed in\n/mit/$USER/web_scripts/$addrend.\n"; + } + + open(VERSION, ">.scripts-version") or die "Can't write scripts-version file: $!\n"; + print VERSION strftime("%F %T %z\n", localtime); + print VERSION $ENV{'USER'}, '@', getclienthostname(), "\n"; + my $tarball = abs_path("/mit/scripts/deploy$scriptsdev/$deploy.tar.gz"); + print VERSION $tarball, "\n"; + $tarball =~ s|/deploydev/|/deploy/|; + print VERSION dirname($tarball), "\n"; + close(VERSION); + + select STDOUT; + $| = 1; # STDOUT is *hot*! +} + +1; Index: branches/fc20-dev/locker/deploy/bin/onserver_star.pm =================================================================== --- branches/fc20-dev/locker/deploy/bin/onserver_star.pm (revision 2523) +++ branches/fc20-dev/locker/deploy/bin/onserver_star.pm (revision 2523) @@ -0,0 +1,142 @@ +package onserver_star; +use strict; +use Exporter; +use Sys::Hostname; +use File::Spec::Functions; +use File::Basename; +use Socket; +use Cwd qw(abs_path); +use POSIX qw(strftime); +use LWP::UserAgent; +use URI; +our @ISA = qw(Exporter); +our @EXPORT = qw(setup totmp fetch_uri print_login_info press_enter $server $tmp $USER $HOME $sname $deploy $addrend $base_uri $ua $admin_username $requires_sql $addrlast $sqlhost $sqluser $sqlpass $sqldb $admin_password $scriptsdev $human $email); + +our $server = "scripts.mit.edu"; + +our ($tmp, $USER, $HOME, $sname, $deploy, $addrend, $base_uri, $ua, $admin_username, $requires_sql, $addrlast, $sqlhost, $sqluser, $sqlpass, $sqldb, $admin_password, $scriptsdev, $human, $email); + +$tmp = ".scripts-tmp"; +sub totmp { + open(FILE, ">$tmp"); + print FILE $_[0]; + close(FILE); +} + +$ua = LWP::UserAgent->new; +push @{$ua->requests_redirectable}, 'POST'; + +sub fetch_uri { + my ($uri, $get, $post) = @_; + my $u = URI->new($uri); + my $req; + if (defined $post) { + $u->query_form($post); + my $content = $u->query; + $u->query_form($get); + $req = HTTP::Request->new(POST => $u->abs($base_uri)); + $req->content_type('application/x-www-form-urlencoded'); + $req->content($content); + } else { + $u->query_form($get) if (defined $get); + $req = HTTP::Request->new(GET => $u->abs($base_uri)); + } + my $res = $ua->request($req); + if ($res->is_success) { + return $res->content; + } else { + print STDERR "Error fetching configuration page: ", $res->status_line, "\n"; + return undef; + } +} + +sub print_login_info { + print "\nYou will be able to log in to $sname using the following:\n"; + print " username: $admin_username\n"; + print " password: $admin_password\n"; +} + +sub getclienthostname { + if (my $sshclient = $ENV{"SSH_CLIENT"}) { + my ($clientip) = split(' ', $sshclient); + my $hostname = gethostbyaddr(inet_aton($clientip), AF_INET); + return $hostname || $clientip; + } else { + return hostname(); + } +} + +sub press_enter { + local $/ = "\n"; + print "Press [enter] to continue with the install."; + my $enter = ; +} + +sub setup { + $ENV{PATH} = '/bin:/usr/bin'; + $USER = $ENV{USER}; + $HOME = $ENV{HOME}; + + ($sname, $deploy, $addrend, $admin_username, $requires_sql, $scriptsdev, $human) = @ARGV; + chdir "$HOME/web_scripts/$addrend"; + $email = "$human\@mit.edu"; + + if($addrend =~ /^(.*)\/$/) { + $addrend = $1; + } + ($addrlast) = ($addrend =~ /([^\/]*)$/); + + $base_uri = "http://$USER.$server/$addrend/"; + + if($requires_sql) { + print "\nCreating SQL database for $sname...\n"; + + open GETPWD, '-|', "/mit/scripts/sql/bin$scriptsdev/get-password"; + ($sqlhost, $sqluser, $sqlpass) = split(/\s/, ); + close GETPWD; + open SQLDB, '-|', "/mit/scripts/sql/bin$scriptsdev/get-next-database", $addrlast; + $sqldb = ; + close SQLDB; + open SQLDB, '-|', "/mit/scripts/sql/bin$scriptsdev/create-database", $sqldb; + $sqldb = ; + close SQLDB; + if($sqldb eq "") { + print "\nERROR:\n"; + print "Your SQL account failed to create a SQL database.\n"; + print "You should log in at http://sql.mit.edu to check whether\n"; + print "your SQL account is at its database limit or its storage limit.\n"; + print "If you cannot determine the cause of the problem, please\n"; + print "feel free to contact sql\@mit.edu for assistance.\n"; + open FAILED, ">.failed"; + close FAILED; + exit 1; + } + } + + if(-e "$HOME/web_scripts/$addrend/.admin") { + open ADMIN, "<$HOME/web_scripts/$addrend/.admin"; + $admin_password=; + chomp($admin_password); + close ADMIN; + unlink "$HOME/web_scripts/$addrend/.admin"; + } + + print "\nConfiguring $sname...\n"; + if($requires_sql) { + print "A copy of ${USER}'s SQL login info will be placed in\n/mit/$USER/web_scripts/$addrend.\n"; + } + + open(VERSION, ">.scripts-version") or die "Can't write scripts-version file: $!\n"; + print VERSION strftime("%F %T %z\n", localtime); + print VERSION $ENV{'USER'}, '@', getclienthostname(), "\n"; + my $tarball = abs_path("/mit/scripts/deploy$scriptsdev/$deploy.tar.gz"); + print VERSION $tarball, "\n"; + $tarball =~ s|/deploydev/|/deploy/|; + print VERSION dirname($tarball), "\n"; + close(VERSION); + + select STDOUT; + $| = 1; # STDOUT is *hot*! +} + +1; Index: branches/fc20-dev/locker/deploy/bin/phpbb =================================================================== --- branches/fc20-dev/locker/deploy/bin/phpbb (revision 2523) +++ branches/fc20-dev/locker/deploy/bin/phpbb (revision 2523) @@ -0,0 +1,203 @@ +#!/usr/bin/perl +use strict; +use FindBin qw($Bin); +use lib $Bin; +use onserver_star; +use File::Path; + +setup(); + +# Initial installation page +fetch_uri( + 'install/index.php', + {mode => 'install', + language => 'en'}, + {}); +print "Loaded initial installation page\n"; + +# Requirements page +fetch_uri( + 'install/index.php', + {mode => 'install', + sub => 'requirements', + language => 'en'}, + {}); +print "Loaded requirements page\n"; + +# Database page +fetch_uri( + 'install/index.php', + {mode => 'install', + sub => 'database', + language => 'en'}, + {img_imagick => '/usr/bin/'}); +print "Loaded database page\n"; + +# Database again +fetch_uri( + 'install/index.php', + {mode => 'install', + sub => 'database'}, + {dbms => 'mysql', + dbhost => $sqlhost, + dbname => $sqldb, + dbuser => $sqluser, + dbpasswd => $sqlpass, + table_prefix => '', + img_imagick => '/usr/bin/', + language => 'en', + testdb => 'true'}); +print "Loaded database page (part 2)\n"; + +# Administrator page +fetch_uri( + 'install/index.php', + {mode => 'install', + sub => 'administrator'}, + {img_imagick => '/usr/bin/', + language => 'en', + dbms => 'mysql', + dbhost => $sqlhost, + dbport => '', + dbname => $sqldb, + dbuser => $sqluser, + dbpasswd => $sqlpass, + table_prefix => ''}); +print "Loaded administrator page\n"; + +# Administrator page again +fetch_uri( + 'install/index.php', + {mode => 'install', + sub => 'administrator'}, + {default_lang => 'en', + admin_name => $admin_username, + admin_pass1 => $admin_password, + admin_pass2 => $admin_password, + board_email1 => $email, + board_email2 => $email, + img_imagick => '/usr/bin/', + language => 'en', + dbms => 'mysql', + dbhost => $sqlhost, + dbport => '', + dbname => $sqldb, + dbuser => $sqluser, + dbpasswd => $sqlpass, + table_prefix => '', + check => 'true'}); +print "Loaded administrator page (part 2)\n"; + +# Config file page +fetch_uri( + 'install/index.php', + {mode => 'install', + sub => 'config_file'}, + {img_imagick => '/usr/bin/', + default_lang => 'en', + admin_name => $admin_username, + admin_pass1 => $admin_password, + admin_pass2 => $admin_password, + board_email1 => $email, + board_email2 => $email, + language => 'en', + dbms => 'mysql', + dbhost => $sqlhost, + dbport => '', + dbname => $sqldb, + dbuser => $sqluser, + dbpasswd => $sqlpass, + table_prefix => ''}); +print "Loaded config file page\n"; + +# Advanced page +fetch_uri( + 'install/index.php', + {mode => 'install', + sub => 'advanced'}, + {img_imagick => '/usr/bin/', + language => 'en', + dbms => 'mysql', + dbhost => $sqlhost, + dbport => '', + dbname => $sqldb, + dbuser => $sqluser, + dbpasswd => $sqlpass, + table_prefix => '', + default_lang => 'en', + admin_name => $admin_username, + admin_pass1 => $admin_password, + admin_pass2 => $admin_password, + board_email1 => $email, + board_email2 => $email}); +print "Loaded advanced setup page\n"; + +# Create database tables +fetch_uri( + 'install/index.php', + {mode => 'install', + sub => 'create_table'}, + {email_enable => '1', + smtp_delivery => '0', + smtp_auth => 'PLAIN', + cookie_secure => '0', + force_server_vars => '0', + server_protocol => 'http://', + server_name => "$USER.scripts.mit.edu", + server_port => '80', + script_path => "/$addrend", + img_imagick => '/usr/bin/', + language => 'en', + dbms => 'mysql', + dbhost => $sqlhost, + dbport => '', + dbname => $sqldb, + dbuser => $sqluser, + dbpasswd => $sqlpass, + table_prefix => '', + default_lang => 'en', + admin_name => $admin_username, + admin_pass1 => $admin_password, + admin_pass2 => $admin_password, + board_email1 => $email, + board_email2 => $email}); +print "Loaded database table creation page\n"; + +# Final page +fetch_uri( + 'install/index.php', + {mode => 'install', + sub => 'final'}, + {language => 'en', + dbms => 'mysql', + dbhost => $sqlhost, + dbport => '', + dbuser => $sqluser, + dbpasswd => $sqlpass, + dbname => $sqldb, + table_prefix => '', + default_lang => 'en', + admin_name => $admin_username, + admin_pass1 => $admin_password, + admin_pass2 => $admin_password, + board_email1 => $email, + board_email2 => $email, + img_imagick => '/usr/bin/', + ftp_path => '', + ftp_user => '', + ftp_pass => '', + email_enable => '1', + smtp_delivery => '0', + smtp_host => '', + smtp_auth => 'PLAIN', + smtp_user => '', + smtp_pass => '', + cookie_secure => '0', + force_server_vars => '0', + server_protocol => 'http://', + server_name => "$USER.scripts.mit.edu", + server_port => '80', + script_path => "/$addrend"}); +print "Loaded installation finalization page\n"; + +rmtree(['install']); Index: branches/fc20-dev/locker/deploy/bin/prompt-password =================================================================== --- branches/fc20-dev/locker/deploy/bin/prompt-password (revision 2523) +++ branches/fc20-dev/locker/deploy/bin/prompt-password (revision 2523) @@ -0,0 +1,40 @@ +#!/bin/sh + +name=$1 +deploy=$2 +addrend=$3 +admin_username=$4 +cd $HOME/web_scripts/$addrend + +echo +echo You will be able to control your copy of $name by logging in to +echo $name using username \"$admin_username\". This account will have a +echo password of your choice. You should not use your Athena account password. + +done="0" +while [ "$done" = "0" ] +do + echo "Please decide upon an admin password, type it, and press [enter]." + echo "This password may only contain a-z, A-Z, and 0-9." + printf "As you type your password, the cursor will not move: " + read admin_pass1 + echo + echo + echo "Now please type the same password again and press [enter]." + printf "As you type your password, the cursor will not move: " + read admin_pass2 + echo + + if [ "$admin_pass1" = "$admin_pass2" ]; then + done="1" + else + echo + echo == Sorry, those passwords do not match. Please try again. == + fi +done + +echo +echo OK. Continuing with the install... +cat < $HOME/web_scripts/$addrend/.admin +$admin_pass1 +EOF Index: branches/fc20-dev/locker/deploy/bin/rails =================================================================== --- branches/fc20-dev/locker/deploy/bin/rails (revision 2523) +++ branches/fc20-dev/locker/deploy/bin/rails (revision 2523) @@ -0,0 +1,274 @@ +#!/usr/bin/perl +use strict; +use FindBin qw($Bin); +use lib $Bin; +use onserver; +use Tie::File; +use Cwd; + +setup(); + +sub make_db { + my($type) = @_; + print "\nCreating $type SQL database for $sname...\n"; + open GETPWD, '-|', "/mit/scripts/sql/bin$scriptsdev/get-password"; + ($sqlhost, $sqluser, $sqlpass) = split(/\s/, ); + close GETPWD; + open SQLDB, '-|', "/mit/scripts/sql/bin$scriptsdev/get-next-database", "${addrlast}_${type}"; + $sqldb = ; + close SQLDB; + open SQLDB, '-|', "/mit/scripts/sql/bin$scriptsdev/create-database", $sqldb; + $sqldb = ; + close SQLDB; + if($sqldb eq "") { + print "\nERROR:\n"; + print "Your SQL account failed to create a SQL database.\n"; + print "You should log in at http://sql.mit.edu to check whether\n"; + print "your SQL account is at its database limit or its storage limit.\n"; + print "If you cannot determine the cause of the problem, please\n"; + print "feel free to contact sql\@mit.edu for assistance.\n"; + exit 1; + } + return $sqldb; +} + +my $dev_db = make_db("development"); +my $test_db = make_db("test"); +my $prod_db = make_db("production"); + +my $cwd = getcwd; +system("rails", "new", $cwd ,"-d", "mysql"); +my $appdir = `basename $cwd`; +chomp $appdir; + +open APPLICATION_RB, "config/application.rb"; +my $appclass; +while() { + if (/module (\w+)\n/) { + $appclass = $1; + last; + } +} +close APPLICATION_RB; +if (!$appclass) { + die "Couldn't find application class name - plase email scripts\@mit.edu with the names of your locker and the application you tried to create. Sorry!"; +} + +open PUBLIC_HTACCESS, ">public/.htaccess"; +print PUBLIC_HTACCESS <.htaccess"; +print HTACCESS </; +} +untie @railswelcome; + +# set config.action_controller.asset_host for all environments, +# so urls to static assets are generated correctly +# regardless of how the app is accessed +my $rails_assethost = " config.action_controller.asset_host = \"//$USER.scripts.mit.edu/$appdir/public\""; +my @environments = ('development', 'production', 'test'); + +for my $environment (@environments) { + tie my @envfile, 'Tie::File', "config/environments/$environment.rb"; + my $i = 0; + for (@envfile) { + if (/^end$/) { + last; + } + ++$i; + } + splice @envfile, $i, 1, ($rails_assethost, 'end'); + untie @envfile; +} + + +tie my @railsfcgi, 'Tie::File', 'public/dispatch.fcgi'; +for (@railsfcgi) { + s/^[^#]*RailsFCGIHandler/## Commented out by scripts.mit.edu autoinstaller\n## RailsFCGIHandler/; +} +untie @railsfcgi; +open RAILSFCGI, ">>public/dispatch.fcgi"; +print RAILSFCGI "#!/usr/bin/ruby\n"; +print RAILSFCGI < ${appclass}::Application)) + rescue => e + dispatch_logger.error(e) + raise e + end +end +t2 = Thread.new do + # List of directories to watch for changes before reload. + # You may want to also watch public or vendor, depending on your needs. + Thread.current[:watched_dirs] = ['app', 'config', 'db', 'lib'] + + # List of specific files to watch for changes. + Thread.current[:watched_files] = ['public/dispatch.fcgi', + 'public/.htaccess'] + # Sample filter: /(\.rb|\.erb)\$/. Default filter: watch all files + Thread.current[:watched_extensions] = // + # Iterations since last reload + Thread.current[:iterations] = 0 + + def modified(file) + begin + mtime = File.stat(file).mtime + rescue + false + else + if Thread.current[:iterations] == 0 + Thread.current[:modifications][file] = mtime + end + Thread.current[:modifications][file] != mtime + end + end + + # Don't symlink yourself into a loop. Please. Things will still work + # (Linux limits your symlink depth) but you will be sad + def modified_dir(dir) + Dir.new(dir).each do |file| + absfile = File.join(dir, file) + if FileTest.directory? absfile + next if file == '.' or file == '..' + return true if modified_dir(absfile) + else + return true if Thread.current[:watched_extensions] =~ absfile && + modified(absfile) + end + end + false + end + + def reload + Thread.current[:modifications] = {} + Thread.current[:iterations] = 0 + # This is a kludge, but at the same time it works. + # Will kill the current FCGI process so that it is reloaded + # at next request. + raise RuntimeError + end + + Thread.current[:modifications] = {} + # Wait until the modify time changes, then reload. + while true + dir_modified = Thread.current[:watched_dirs].inject(false) {|z, dir| z || modified_dir(File.join(File.dirname(__FILE__), '..', dir))} + file_modified = Thread.current[:watched_files].inject(false) {|z, file| z || modified(File.join(File.dirname(__FILE__), '..', file))} + reload if dir_modified || file_modified + Thread.current[:iterations] += 1 + sleep 1 + end +end + +t1.join +t2.join +## End of scripts.mit.edu autoinstaller additions +EOF +chmod 0755,'public/dispatch.fcgi'; + +# static-cat doesn't whitelist .txt files +chmod 0777, 'public/robots.txt'; + +# have to explicitly take a dependency on fcgi +# ruby1.9 means we need to take a dependency on minitest +# for rails console to work +open GEMFILE, ">>Gemfile"; +print GEMFILE "gem 'fcgi'\n"; +print GEMFILE "gem 'minitest'\n"; +close GEMFILE; + +print "Your application is located in:\n"; +print " /mit/$USER/web_scripts/$addrend/\n"; +print "To run programs like rake or rails generate, run\n"; +print " 'ssh -k $USER\@scripts' and cd to the above directory.\n\n"; +press_enter; + +exit 0; Index: branches/fc20-dev/locker/deploy/bin/scripts-chipmunkpoll =================================================================== --- branches/fc20-dev/locker/deploy/bin/scripts-chipmunkpoll (revision 2523) +++ branches/fc20-dev/locker/deploy/bin/scripts-chipmunkpoll (revision 2523) @@ -0,0 +1,6 @@ +#!/bin/sh + +sname="Chipmunk Poll" +deploy="chipmunkpoll" +prompt_username=1 +. /mit/scripts/deploy/bin$scriptsdev/install-onathena Index: branches/fc20-dev/locker/deploy/bin/scripts-exponent =================================================================== --- branches/fc20-dev/locker/deploy/bin/scripts-exponent (revision 2523) +++ branches/fc20-dev/locker/deploy/bin/scripts-exponent (revision 2523) @@ -0,0 +1,6 @@ +#!/bin/sh + +sname="Exponent" +deploy="exponent" +prompt_username=1 +. /mit/scripts/deploy/bin$scriptsdev/install-onathena Index: branches/fc20-dev/locker/deploy/bin/trac =================================================================== --- branches/fc20-dev/locker/deploy/bin/trac (revision 2523) +++ branches/fc20-dev/locker/deploy/bin/trac (revision 2523) @@ -0,0 +1,69 @@ +#!/usr/bin/perl +use strict; +use FindBin qw($Bin); +use lib $Bin; +use onserver; +use Cwd; +use File::Path; +use URI::Escape; +use DBI; +use Config::IniFiles; + +setup(); + +print "\nEnter the name of your project (the title of this Trac instance).\n"; +print "Project name: "; +my $name=; +chomp($name); + +my $dbh = DBI->connect("DBI:mysql:database=$sqldb;host=$sqlhost", $sqluser, $sqlpass, {RaiseError => 1}); +$dbh->do('alter database collate utf8_general_ci'); + +my $dbstring = "mysql://" . uri_escape($sqluser) . ":" . uri_escape($sqlpass) . "\@$sqlhost/$sqldb"; + +print "\nEnter the type of version-control repository this project uses.\n"; +print "You'll have to set up the repo yourself; feel free to ask scripts@ for help.\n"; +print "If you don't want version-control integration, take the default.\n"; +print "Repository type (default svn; also bzr, git, hg): "; +my $repotype=; +chomp($repotype); +$repotype = $repotype ? $repotype : 'svn'; + +print "\nEnter the path to the version-control repository.\n"; +print "If you don't want version-control integration, leave blank.\n"; +print "Path to repository: "; +my $repopath=; +chomp($repopath); + +print STDERR "running trac-admin:\n"; +system(qw(/usr/bin/trac-admin tracdata initenv), + $name, $dbstring, $repotype, $repopath); +# XXX this exposes the SQL password on the command line + +#aka perl -pe 's/\@ADDREND\@/$addrend/g' <.htaccess.in >.htaccess +open IN, '<.htaccess.in'; open OUT, '>.htaccess'; +while () { + s/\@ADDREND\@/~$USER\/$addrend/g; + print OUT $_; +} +close IN; close OUT; + +my $cfg = Config::IniFiles->new(-file => 'tracdata/conf/trac.ini'); +$cfg->setval('trac', 'default_charset', 'utf-8'); +$cfg->AddSection('components'); +$cfg->newval('components', 'webadmin.*', 'enabled'); +$cfg->newval('components', 'tracext.git.*', 'enabled') if $repotype eq "git"; +$cfg->newval('components', 'tracext.hg.*', 'enabled') if $repotype eq "hg"; +$cfg->RewriteConfig(); + +system(qw(/usr/bin/trac-admin tracdata permission add), $human, 'TRAC_ADMIN'); + +chmod 0777, '.htaccess'; +unlink '.htaccess.in'; + +open OUT, '>tracdata/.htaccess'; +print OUT "Deny from all\n"; +close OUT; +chmod 0777, 'tracdata/.htaccess'; + +exit 0; Index: branches/fc20-dev/locker/deploy/bin/turbogears =================================================================== --- branches/fc20-dev/locker/deploy/bin/turbogears (revision 2523) +++ branches/fc20-dev/locker/deploy/bin/turbogears (revision 2523) @@ -0,0 +1,189 @@ +#!/usr/bin/perl +use strict; +use FindBin qw($Bin); +use lib $Bin; +use onserver; +use Cwd; +use File::Path; +use URI::Escape; +use DBI; +use Config::IniFiles; +use FileHandle; + +setup(); + +print "\nEnter the name of your project (the title of this TurboGears instance).\n"; +my $name; +while (1) { + print "Project name: "; + $name=; + chomp($name); + if ($name =~ /^[a-zA-Z][a-zA-Z0-9_ -]+$/) { + last; + } + print "Invalid project name; it should start with a letter and not contain\npunctuation other than dashes or underscores.\n"; +} + +# quickstart turns spaces or underscores into dashes... +$name =~ s/[ _-]+/-/g; + +my $defpack=lc($name); +$defpack =~ s/[ -]/_/g; +$defpack =~ s/[^a-z0-9_]//g; +if (! ($defpack =~ /^[a-zA-Z]/)) { + $defpack = "p$defpack"; +} +print "\nEnter the name for your project's python package.\n"; +my $pack; +while (1) { + print "Package name [${defpack}]: "; + $pack=; + chomp($pack); + if (!($pack)) { + $pack=$defpack; + last; + } elsif ($pack =~ /^[a-zA-Z][a-zA-Z0-9_]+$/) { + last; + } + print "Invalid package name; it should start with a letter and contain only letters,\nnumbers, and underscores.\n"; +} + +print "\nWhat ORM (Object-Relational Mapper) do you want to use with this TurboGears\ninstance? Select from the following list:\n"; +print "1. SQLAlchemy Elixir\n"; +print "2. SQLAlchemy\n"; +print "3. SQLObject\n"; +my $orm; +while (1) { + print "ORM [1]: "; + my $ormnum=; + chomp($ormnum); + if ((!$ormnum) || $ormnum == 1) { + $orm = "elixir"; + last; + } elsif ($ormnum == 2) { + $orm = "sqlalchemy"; + last; + } elsif ($ormnum == 3) { + $orm = "sqlobject"; + last; + } + print "Please choose 1, 2, or 3.\n"; +} + +print "\nWhat template do you want to use with this TurboGears instance? Select from\nthe following list:\n"; +print "1. turbogears: normal template, recommended for most projects\n"; +print "2. tgbig: a more complex directory structure for big projects\n"; +my $templ; +while (1) { + print "Template [1]: "; + my $templnum=; + chomp($templnum); + if ((!$templnum) || $templnum == 1) { + $templ = "turbogears"; + last; + } elsif ($templnum == 2) { + $templ = "tgbig"; + last; + } + print "Please choose 1, 2, or 3.\n"; +} + +print "\nDo you want to use Identity (usernames/passwords) in this project?\n(These would be separate from Athena usernames/passwords.)\n"; +print "1. no identity: no logins, everyone sees the same pages\n"; +print "2. standard identity: users log in with site-specific usernames and passwords\n"; +#print "3. certificates: users are identified by their MIT certificates\n"; +my $ident; +my $certpatch=0; +while (1) { + print "Identity [1]: "; + my $identnum=; + chomp($identnum); + if ((!$identnum) || $identnum == 1) { + $ident = "no"; + last; + } elsif ($identnum == 2) { + $ident = "yes"; + last; + } elsif ($identnum == 3) { + $ident = "yes"; + $certpatch = 1; + last; + } +} + +open (FLUPCONF, ">flupconfig.py"); +print FLUPCONF <autoflush(1); +print QS "$ident\n" or die("tg-admin quickstart failed specify ident!"); +close(QS) or die("tg-admin quickstart failed close!"); + +# Put in the sqldb +if ($orm eq "sqlobject") { + my $uriuser = uri_escape($sqluser); + my $uripass = uri_escape($sqlpass); + foreach my $fil (("$name/dev.cfg", "$name/sample-prod.cfg")) + { + open my $in, '<', $fil or die "Can't read old file: $!"; + open my $out, '>', "$fil.new" or die "Can't write new file: $!"; + + while (<$in>) { + s/^sqlobject\.dburi(.*)$/#sqlobject.dburi\2\nsqlobject.dburi="mysql:\/\/$uriuser:$uripass\@$sqlhost\/$sqldb"/; + print $out $_; + } + close $out; + rename "$fil.new", $fil + or die "Cannot rename: $!"; + } +} else { + system(qw(sed -ri),"s&^sql(alchemy|object)\.dburi(.*)\$&#sql\\1.dburi\\2\\nsql\\1.dburi=\"mysql://$sqlhost/$sqldb?read_default_file=~/.my.cnf\"&","$name/dev.cfg", "$name/sample-prod.cfg") == 0 or die "sed db failed!"; +} +system(qw(sed -ri),'s/^#? *autoreload\.on.*$/autoreload.on = False # breaks the scripts flup setup/',"$name/dev.cfg") == 0 or die "sed autoreload failed!"; +my $addrendescsl = $addrend; +$addrendescsl =~ s|/|\\/|g; +# Obviated by a TurboGears upgrade +#system(qw(sed -ri),'s/^(\[global\] *)$/\1\nserver.webpath = "\/'."$addrendescsl".'"/',"$name/dev.cfg") == 0 or die "sed webpath failed!"; +if ($orm eq "elixir" or $orm eq "sqlalchemy") { + system(qw(sed -ri),'s/^(\[global\] *)$/\1\nsqlalchemy.pool_recycle = 30 # Need a short timeout for sql.mit.edu/',"$name/$pack/config/app.cfg") == 0 or die "sed pool_recycle failed!"; +} + +# Make logdir +system('mkdir','-p',"$name/log"); + +# Cert patch +if ($certpatch) { + # comment out the password = line in model + system(qw(sed -ri), + 's/^(.*password.*)$/#\1 -- we use certs, not passwords/', + "$name/$pack/model.py") == 0 or die "sed model for certs failed!"; + + # Stick cert.py in + system('cp',"/mit/scripts/deploy$scriptsdev/turbogears-certs/certs.py", + "$name/$pack/") == 0 or die "cp certs.py failed!"; + + # Add the certness to controllers.py + system(qw(sed -ri), + 's/^(from cherrypy.*)$/\1\nfrom '."$pack".'.certs import with_mit_certs/', + "$name/$pack/controllers.py") == 0 or die "sed controllers import for certs failed!"; + system(qw(sed -ri), + 's/^(\s+)(def login.*)$/\1@with_mit_certs\n\1\2', + "$name/$pack/model.py") == 0 or die "sed model for certs failed!"; + #-! replace login body + #-! replace logout body + #-! replace login.kid +} + +exit 0; Index: branches/fc20-dev/locker/deploy/bin/wordpress =================================================================== --- branches/fc20-dev/locker/deploy/bin/wordpress (revision 2523) +++ branches/fc20-dev/locker/deploy/bin/wordpress (revision 2523) @@ -0,0 +1,42 @@ +#!/usr/bin/perl +use strict; +use FindBin qw($Bin); +use lib $Bin; +use onserver; +use DBI; + +setup(); + +fetch_uri( + 'wp-admin/setup-config.php', + {step => 2}, + {dbhost => $sqlhost, + uname => $sqluser, + dbname => $sqldb, + pwd => $sqlpass, + prefix => '', + submit => 'Submit', + step => 2}); + +my $html = fetch_uri( + 'wp-admin/install.php', + {step => 2}, + {weblog_title => 'My Blog', + admin_email => $email, + submit => 'Continue', + step => 2}); +while ($html =~ /(.*)<\/code>/g) { + $admin_username = $admin_password; + $admin_password = $1; +} + +my $dbh = DBI->connect("dbi:mysql:database=$sqldb;host=$sqlhost", $sqluser, $sqlpass); +$dbh->do("update wp_options set option_value = ? where option_name = 'siteurl'", + {}, "/~$USER/$addrend"); +$dbh->do("update wp_options set option_value = ? where option_name = 'home'", + {}, "http://$USER.$server/$addrend"); +$dbh->disconnect; + +print_login_info(); +print "You will also receive this login information at your MIT email address.\n"; +press_enter(); Index: branches/fc20-dev/locker/deploy/mediawiki-1.11.0/mediawiki-1.11.0.patch =================================================================== --- branches/fc20-dev/locker/deploy/mediawiki-1.11.0/mediawiki-1.11.0.patch (revision 2523) +++ branches/fc20-dev/locker/deploy/mediawiki-1.11.0/mediawiki-1.11.0.patch (revision 2523) @@ -0,0 +1,24 @@ +diff -Nur mediawiki-1.11.0.old/AdminSettings.php mediawiki-1.11.0/AdminSettings.php +--- mediawiki-1.11.0.old/AdminSettings.php 1969-12-31 19:00:00.000000000 -0500 ++++ mediawiki-1.11.0/AdminSettings.php 2007-10-09 17:32:52.000000000 -0400 +@@ -0,0 +1,20 @@ ++ Index: branches/fc20-dev/locker/deploy/mediawiki-1.5.6/mediawiki-1.5.6.patch =================================================================== --- branches/fc20-dev/locker/deploy/mediawiki-1.5.6/mediawiki-1.5.6.patch (revision 2523) +++ branches/fc20-dev/locker/deploy/mediawiki-1.5.6/mediawiki-1.5.6.patch (revision 2523) @@ -0,0 +1,15 @@ +--- includes/Setup.php Fri Aug 26 10:05:43 2005 ++++ includes/Setup.php Fri Sep 23 18:46:34 2005 +@@ -117,12 +117,6 @@ + wfProfileOut( $fname.'-memcached' ); + wfProfileIn( $fname.'-SetupSession' ); + +-if ( $wgDBprefix ) { +- session_name( $wgDBname . '_' . $wgDBprefix . '_session' ); +-} else { +- session_name( $wgDBname . '_session' ); +-} +- + if( !$wgCommandLineMode && ( isset( $_COOKIE[session_name()] ) || isset( $_COOKIE[$wgDBname.'Token'] ) ) ) { + User::SetupSession(); + $wgSessionStarted = true; Index: branches/fc20-dev/locker/deploy/mediawiki-1.5.8/mediawiki-1.5.8.patch =================================================================== --- branches/fc20-dev/locker/deploy/mediawiki-1.5.8/mediawiki-1.5.8.patch (revision 2523) +++ branches/fc20-dev/locker/deploy/mediawiki-1.5.8/mediawiki-1.5.8.patch (revision 2523) @@ -0,0 +1,17 @@ +diff -uNr mediawiki-1.5.8/includes/Setup.php mw-scripts-158/includes/Setup.php +--- includes/Setup.php 2006-02-11 02:26:47.000000000 -0500 ++++ includes/Setup.php 2006-04-14 18:28:29.000000000 -0400 +@@ -118,11 +118,11 @@ + wfProfileIn( $fname.'-SetupSession' ); + + if ( $wgDBprefix ) { +- $wgCookiePrefix = $wgDBname . '_' . $wgDBprefix; ++ $wgCookiePrefix = str_replace("+", "", $wgDBname . '_' . $wgDBprefix); + } elseif ( $wgSharedDB ) { + $wgCookiePrefix = $wgSharedDB; + } else { +- $wgCookiePrefix = $wgDBname; ++ $wgCookiePrefix = str_replace("+", "", $wgDBname); + } + + session_name( $wgCookiePrefix . '_session' ); Index: branches/fc20-dev/locker/deploy/trac/Makefile =================================================================== --- branches/fc20-dev/locker/deploy/trac/Makefile (revision 2523) +++ branches/fc20-dev/locker/deploy/trac/Makefile (revision 2523) @@ -0,0 +1,5 @@ +trac.tar.gz: trac + tar czf $@ trac + +install: trac.tar.gz + cp $^ .. Index: branches/fc20-dev/locker/deploy/trac/trac.fcgi =================================================================== --- branches/fc20-dev/locker/deploy/trac/trac.fcgi (revision 2523) +++ branches/fc20-dev/locker/deploy/trac/trac.fcgi (revision 2523) @@ -0,0 +1,70 @@ +#!/usr/bin/python + +import os, os.path, sys +from trac.web.main import dispatch_request +from trac.web._fcgi import WSGIServer +import urlparse + +env_path = os.getcwd()+'/tracdata' +os.environ['TRAC_ENV'] = env_path + +def send_upgrade_message(environ, start_response): + import pwd + start_response('500 Internal Server Error', []) + locker = pwd.getpwuid(os.getuid())[0] + return ['''This Trac instance needs to be upgraded. + +From an Athena machine, type + ssh %s@scripts trac-admin %s upgrade --no-backup + ssh %s@scripts trac-admin %s wiki upgrade +to upgrade, and then + add scripts + for-each-server -l %s pkill -u %s trac.fcgi +to get this message out of the way. + +Please ask the scripts.mit.edu maintainers for help +if you have any trouble, at scripts@mit.edu. +''' % (locker, env_path, locker, env_path, locker, locker)] + +def setup_env(): + '''Obtain the environment, handling the needs-upgrade check, and cache it. + + This mimics open_environment in trac/env.py.''' + import trac.env + env = trac.env.Environment(env_path) + needs_upgrade = False + try: + needs_upgrade = env.needs_upgrade() + except Exception, e: # e.g. no database connection + env.log.exception(e) + if env.needs_upgrade(): + WSGIServer(send_upgrade_message).run() + sys.exit(0) + if hasattr(trac.env, 'env_cache'): + trac.env.env_cache[env_path] = env +setup_env() + +def my_dispatch_request(environ, start_response): + if ('REDIRECT_URL' in environ and 'PATH_INFO' in environ + and environ['REDIRECT_URL'].endswith(environ['PATH_INFO'])): + environ['SCRIPT_NAME'] = environ['REDIRECT_URL'][:-len(environ['PATH_INFO'])] + + # If the referrer has our hostname and path, rewrite it to have + # the right protocol and port, too. This lets the login link go + # to the right page. + if 'HTTP_REFERER' in environ: + referrer = urlparse.urlsplit(environ['HTTP_REFERER']) + base = urlparse.urlsplit( + ('https://' if environ.get('HTTPS') == 'on' else 'http://') + + environ['HTTP_HOST'] + + environ['SCRIPT_NAME']) + if referrer.hostname == base.hostname and \ + (referrer.path == base.path or + referrer.path.startswith(base.path + '/')): + environ['HTTP_REFERER'] = urlparse.urlunsplit( + (base.scheme, base.netloc, + referrer.path, referrer.query, referrer.fragment)) + + return dispatch_request(environ, start_response) + +WSGIServer(my_dispatch_request).run() Index: branches/fc20-dev/locker/deploy/trac/trac/.htaccess.in =================================================================== --- branches/fc20-dev/locker/deploy/trac/trac/.htaccess.in (revision 2523) +++ branches/fc20-dev/locker/deploy/trac/trac/.htaccess.in (revision 2523) @@ -0,0 +1,19 @@ +AuthType SSLCert +AuthSSLCertAuthoritative off +AuthSSLCertVar SSL_CLIENT_S_DN_Email +AuthSSLCertStripSuffix @MIT.EDU +Require valid-user +AuthOptional on + +RewriteEngine on + +RewriteCond %{HTTPS} =on +RewriteRule ^logout http://%{SERVER_NAME}%{REQUEST_URI} [R,L] + +RewriteCond %{REQUEST_URI} !^/@ADDREND@/trac.fcgi +RewriteRule ^(.*)$ /@ADDREND@/trac.fcgi/$1 [L] + + +AuthOptional off +ErrorDocument 401 /__scripts/needcerts + Index: branches/fc20-dev/locker/deploy/trac/trac/trac.fcgi =================================================================== --- branches/fc20-dev/locker/deploy/trac/trac/trac.fcgi (revision 2523) +++ branches/fc20-dev/locker/deploy/trac/trac/trac.fcgi (revision 2523) @@ -0,0 +1,2 @@ +#!/bin/sh +exec /afs/athena.mit.edu/contrib/scripts/deploy/trac/trac.fcgi "$@" Index: branches/fc20-dev/locker/doc/autoinstallers =================================================================== --- branches/fc20-dev/locker/doc/autoinstallers (revision 2523) +++ branches/fc20-dev/locker/doc/autoinstallers (revision 2523) @@ -0,0 +1,63 @@ +-*- text -*- + +== How to update an autoinstaller to a new version of the upstream package == + +Things to check beforehand: +- Make sure /mit/scripts/deploydev is up to date from /mit/scripts/deploy (deploydev is not a svn checkout and is probably missing changes from deploy) + +Steps: +# Example values +PKG=wordpress +NEWVERS=2.5.1 +OLDVERS=2.3.3 +URL=http://www.example.com/download/wordpress-2.5.1.tar.gz + +# Create a new directory in /mit/scripts/deploydev for the new version +mkdir /mit/scripts/deploydev/$PKG-$NEWVERS + +# Copy any scripts patches to the new directory +cp /mit/scripts/deploydev/$PKG-$OLDVERS/*.patch /mit/scripts/deploydev/$PKG-$NEWVERS/ +# Some scripts might have other files in here +# Make sure the patch is still relevant for the current version of the package + +# Download the new package from the upstream site +wget -O /mit/scripts/deploydev/$PKG-$NEWVERS/$PKG-$NEWVERS.tar.gz "$URL" + +# Update the symlink for the new package version +cd /mit/scripts/deploydev && ln -nsf $PKG-$NEWVERS/$PKG-NEWVERS.tar.gz $PKG.tar.gz + +# Try an autoinstall +ssh linerva -t env scriptsdev=dev athrun scripts + +# Fix any bugs that were introduced, probably in /mit/scripts/deploydev/bin/$PKG or /mit/scripts/bin/scripts-$PKG, or the patches + +# Commit your changes by moving them to /mit/scripts/deploy +cp -a /mit/scripts/deploydev/$PKG-NEWVERS /mit/scripts/deploydev/$PKG.tar.gz /mit/scripts/deploy/ + +# Test the new version from linerva and athena.dialup + + +== How to generate an autoupdate == + +cd /tmp +# Use --dev if you haven't pushed to deploy yet +/mit/scripts/sbin/propose-update --dev $PKG $OLDVERS $NEWVERS +cd $PKG-$OLDVERS-to-$NEWVERS.proposal +# If there is any custom setup that needs to be performed +# (e.g. something in /mit/scripts/deploy/bin), do so to the +# $PKG-$OLDVERS and $PKG-$NEWVERS directories, then do +(cd .. && /mit/scripts/sbin/propose-update --redo-all --dev $PKG $OLDVERS $NEWVERS) +# Look at the files that were generated to make sure they're sane +# If necessary, add pre- and post- hooks, such as for a DB update script +emacs extra/prepatch.sh +emacs extra/postpatch.sh +# Generate the patch +cd .. +# --dev here means to put the update in /mit/scripts/deploydev/updates +/mit/scripts/sbin/build-update --dev $PKG $OLDVERS $NEWVERS +# Test the update by running cd /mit/foo/web_scripts/bar && /mit/scripts/deploydev/updates/$PKG-#OLDVERS-to-$NEWVERS/update "foo" + +# If necessary, repeat this process. You can edit the files in the +# $PKG-$OLDVERS and $PKG-$NEWVERS subdirectories of the proposal, +# passing --redo-* options to propose-update as necessary to cause it +# to regenerate the file lists from the subdirectories Index: branches/fc20-dev/locker/doc/cluedump/AFS.tex =================================================================== --- branches/fc20-dev/locker/doc/cluedump/AFS.tex (revision 2523) +++ branches/fc20-dev/locker/doc/cluedump/AFS.tex (revision 2523) @@ -0,0 +1,65 @@ +\subsection{AFS} + +\begin{frame} + \frametitle{AFS access controls} + \begin{itemize} + \item AFS enforces server side access controls. + \item On Athena systems: user's password $\to$ Kerberos tickets + $\to$ AFS tokens, which authenticate the client to the AFS server. + \item On scripts, we don't have the user's password or tickets. + \item User's scripts are not publicly readable. + \item Access is controlled through a single {\tt daemon.scripts} AFS + user. + \end{itemize} +\end{frame} + +\begin{frame} + \frametitle{Isolating users on scripts} + \begin{itemize} + \item If all users share {\tt daemon.scripts} AFS tokens, how are + they prevented from accessing each other's {\tt web\_scripts}? + \item On scripts, we enforce additional restrictions in the AFS + kernel module. + \begin{itemize} + \item \texttt{afsAccessOK()} in + \texttt{openafs/src/afs/VNOPS/afs\_vnop\_access.c} + \end{itemize} + \end{itemize} +\end{frame} + +\begin{frame}[fragile] + \only<1>{You can only use {\tt daemon.scripts} credentials to access + files in a volume with volume ID equal to your UID,}% + \only<2>{or the file is {\tt system:anyuser} readable + anyway,\vspace{\baselineskip}}% + \only<3>{or the {\tt apache} or {\tt postfix} users are doing a {\tt + stat()},\vspace{\baselineskip}}% + \only<4>{or the {\tt apache} user is trying to read a file with mode + {\tt 777},\vspace{\baselineskip}}% + \only<5>{or the {\tt root} or {\tt signup} users are accessing file + with the special {\tt D} or {\tt E} bits.}% + +\begin{footnotesize} +\begin{semiverbatim} + int + afs_AccessOK(struct vcache *avc, afs_int32 arights, + struct vrequest *areq, afs_int32 check_mode_bits) + \{ + \ldots ++ if (\alert<1>{!(areq->realuid == avc->fid.Fid.Volume)} && ++ \alert<2>{!((avc->anyAccess | arights) == avc->anyAccess)} && ++ \alert<3>{!(arights == PRSFS_LOOKUP && areq->realuid == HTTPD_UID) &&} ++ \alert<3>{!(arights == PRSFS_LOOKUP && areq->realuid == POSTFIX_UID)} && ++ \alert<4>{!(arights == PRSFS_READ && areq->realuid == HTTPD_UID &&} ++ \alert<4>{ avc->m.Mode == 0100777)} && ++ \alert<5>{!(PRSFS_USR3 == afs_GetAccessBits(avc, PRSFS_USR3, areq) &&} ++ \alert<5>{ areq->realuid == 0) &&} ++ \alert<5>{!(PRSFS_USR4 == afs_GetAccessBits(avc, PRSFS_USR4, areq) &&} ++ \alert<5>{ (areq->realuid == 0 || areq->realuid == SIGNUP_UID))}) \{ ++ return 0; ++ \} + \ldots +\end{semiverbatim} +\end{footnotesize} +\end{frame} + Index: branches/fc20-dev/locker/doc/cluedump/LDAP.tex =================================================================== --- branches/fc20-dev/locker/doc/cluedump/LDAP.tex (revision 2523) +++ branches/fc20-dev/locker/doc/cluedump/LDAP.tex (revision 2523) @@ -0,0 +1,17 @@ +\subsection{LDAP} + +\begin{frame} + \frametitle{LDAP data} + + \begin{itemize} + \item All user-specific information is stored in LDAP records + \item Each scripts server runs a local LDAP daemon with multi-master + replication + \item Each user has a \texttt{posixAccount} and at least one + \texttt{apacheConfig} and \texttt{scriptsVhost} + \item Users can request additional virtual hosts + \item We hope to create a web interface (phase 1 of + ``scripts-pony'') for users to create virtual hosts in the + \texttt{*.user.scripts.mit.edu} namespace + \end{itemize} +\end{frame} Index: branches/fc20-dev/locker/doc/cluedump/LVS.tex =================================================================== --- branches/fc20-dev/locker/doc/cluedump/LVS.tex (revision 2523) +++ branches/fc20-dev/locker/doc/cluedump/LVS.tex (revision 2523) @@ -0,0 +1,22 @@ +\subsection{LVS} + +\begin{frame} + \frametitle{Linux Virtual Server} + \begin{itemize} + \item Provides high availability and load balancing + \item {\tt heartbeat} provides failover between LVS ``directors'' + \item {\tt ldirectord} keeps track of online scripts servers and chooses destination server for each request + \end{itemize} +\end{frame} + +\begin{frame} + \frametitle{Load Balancing} + \begin{itemize} + \item Users are assigned to scripts servers based on IP + \item Works around bugs in scripts that assume a single web server + \end{itemize} + \begin{center} + \only<1>{\includegraphics[width=3in] {Aggregated-cps_www-year.png}} + \only<2>{\includegraphics[width=3in] {Aggregated-cps_www-year-clip.png}} + \end{center} +\end{frame} Index: branches/fc20-dev/locker/doc/cluedump/backend.tex =================================================================== --- branches/fc20-dev/locker/doc/cluedump/backend.tex (revision 2523) +++ branches/fc20-dev/locker/doc/cluedump/backend.tex (revision 2523) @@ -0,0 +1,1 @@ +\section{Backend} Index: branches/fc20-dev/locker/doc/cluedump/closing.tex =================================================================== --- branches/fc20-dev/locker/doc/cluedump/closing.tex (revision 2523) +++ branches/fc20-dev/locker/doc/cluedump/closing.tex (revision 2523) @@ -0,0 +1,7 @@ +\section{Further Info} +\begin{frame} + \frametitle{Further Info} + Subversion: {\tt svn://scripts.mit.edu/} + \\ + Scripts Hackathon \\ Saturday, 2 PM, W20-557 +\end{frame} Index: branches/fc20-dev/locker/doc/cluedump/contents.tex =================================================================== --- branches/fc20-dev/locker/doc/cluedump/contents.tex (revision 2523) +++ branches/fc20-dev/locker/doc/cluedump/contents.tex (revision 2523) @@ -0,0 +1,5 @@ +\begin{frame} + \frametitle{Outline} + \tableofcontents[pausesections] +\end{frame} + Index: branches/fc20-dev/locker/doc/cluedump/httpdmods.tex =================================================================== --- branches/fc20-dev/locker/doc/cluedump/httpdmods.tex (revision 2523) +++ branches/fc20-dev/locker/doc/cluedump/httpdmods.tex (revision 2523) @@ -0,0 +1,50 @@ +\subsection{Apache modules} + +\begin{frame}[fragile] + \frametitle{Apache modules} + \begin{itemize} + \item We make it easy to do authentication against MIT certificates. + \item Both \texttt{https://scripts-cert.mit.edu}, and port + \texttt{444} on any scripts hostname, are configured to request + client certificates. + \item \texttt{mod\_ssl} provides the + \texttt{SSL\_CLIENT\_S\_DN\_Email} environment variable, but does + not integrate with the Apache authentication and authorization + framework. + \item Wrote a collection of Apache modules to make this cleaner. + \end{itemize} +\end{frame} + +\begin{frame}[fragile] + \frametitle{\texttt{mod\_auth\_sslcert}} + \begin{itemize} + \item \texttt{mod\_auth\_sslcert} passes the + \texttt{SSL\_CLIENT\_S\_DN\_Email} variable to the Apache + authorization handlers. + \end{itemize} +\begin{semiverbatim} +AuthType SSLCert +AuthSSLCertVar SSL_CLIENT_S_DN_Email +AuthSSLCertStripSuffix "@MIT.EDU" +\end{semiverbatim} +\end{frame} + +\begin{frame}[fragile] + \frametitle{\texttt{mod\_authz\_afsgroup}} + \begin{itemize} + \item \texttt{mod\_authz\_afsgroup} does Apache authorization based + on AFS groups. + \end{itemize} +\begin{semiverbatim} +Require afsgroup system:scripts-team +\end{semiverbatim} +\end{frame} + +\begin{frame}[fragile] + \frametitle{\texttt{mod\_auth\_optional}} + \begin{itemize} + \item \texttt{mod\_auth\_optional} subverts the authorization + process to allow you to serve different pages to users with + certificates and users without certificates. + \end{itemize} +\end{frame} Index: branches/fc20-dev/locker/doc/cluedump/kerberos.tex =================================================================== --- branches/fc20-dev/locker/doc/cluedump/kerberos.tex (revision 2523) +++ branches/fc20-dev/locker/doc/cluedump/kerberos.tex (revision 2523) @@ -0,0 +1,47 @@ +\subsection{Kerberos} + +\begin{frame} + \frametitle{Group locker support} + + \begin{itemize} + \item ``Users'' on scripts are actually lockers. + \item User IDs are actually locker volume IDs. + \pause + \item Kerberos is modified to let users SSH in as any locker they + administrate. + \begin{itemize} + \item Replaced the \texttt{.k5login} mechanism: + \texttt{krb5\_kuserok()} in + \texttt{krb5/src/lib/krb5/os/kuserok.c} + \item Calls a Perl script \texttt{/usr/local/sbin/admof} to do the + actual check. + \end{itemize} + \end{itemize} +\end{frame} + +\begin{frame}[fragile] +\begin{footnotesize} +\begin{semiverbatim} + krb5_boolean KRB5_CALLCONV + krb5_kuserok(krb5_context context, krb5_principal principal, + const char *luser) + \{ + \ldots ++ if ((pid = fork()) == -1) \{ ++ free(princname); ++ return(FALSE); ++ \} ++ if (pid == 0) \{ ++#define ADMOF_PATH "/usr/local/sbin/ssh-admof" ++ exec(ADMOF_PATH, ADMOF_PATH, (char *) luser, princname, NULL); ++ exit(1); ++ \} ++ if (waitpid(pid, &status, 0) > 0 && WIFEXITED(status) && ++ WEXITSTATUS(status) == 33) \{ ++ isok = TRUE; ++ \} + \ldots + \} +\end{semiverbatim} +\end{footnotesize} +\end{frame} Index: branches/fc20-dev/locker/doc/cluedump/services.tex =================================================================== --- branches/fc20-dev/locker/doc/cluedump/services.tex (revision 2523) +++ branches/fc20-dev/locker/doc/cluedump/services.tex (revision 2523) @@ -0,0 +1,89 @@ +\section{Services} + +\subsection{Web} +\begin{frame} + \frametitle{Apache} + \begin{itemize} + \item Everyone wants Apache + \item Apache's default configuration isn't safe for scripting + \item Scripting \emph{requires} code execution---mod\_php, mod\_perl, mod\_python + \item Apache normally runs everything as apache/nobody + \item How to secure? + \pause + \item suEXEC---allows Apache to spawn a process as the user\ldots + \item {\ldots}even for static content! + \end{itemize} +\end{frame} + +\begin{frame} + \frametitle{suEXEC} + \begin{itemize} + \item setuid program + \item Passed the request by Apache + \item Verifies that the script is in the {\tt web\_scripts} directory + \item Switches to the uid of the file and executes + \item Even for static files! + \end{itemize} +\end{frame} + +\subsection{Mail} + +\begin{frame}[fragile] + \frametitle{Postfix} + \begin{itemize} + \item Standard Postfix server + \item No local mailboxes + \item All mail is passed to procmail + \end{itemize} + \begin{verbatim}mailbox_command = /usr/bin/procmail -t \ +-a "${EXTENSION}" ~/mail_scripts/procmailrc\end{verbatim} +\end{frame} + +\begin{frame}[fragile] + \frametitle{procmail} + \begin{itemize} + \item Reads \verb|~/mail_scripts/procmailrc| from user's home directory + \item Users can do whatever they want with messages + \item AFS causes problems---No way to know if failure is temporary (file server is down) or permanent (user isn't signed up for mail scripts) + \item All procmail failures are treated as temporary, so mail is queued + \end{itemize} +\end{frame} + +\subsection{Cron (``Shortjobs'')} + +\begin{frame}[fragile] + \frametitle{Cron (cronie)} + \begin{itemize} + \item Crontabs are currently stored locally on scripts servers + \item {\tt cronload} command loads the crontabs from + \verb|~/cron_scripts/crontab| \pause + \item Needs improvement + \item Cron does not fail over with Web and Mail + \item Plan to move crontabs into AFS and do hot failover + \end{itemize} +\end{frame} + +\subsection{SQL} + +\begin{frame} + \frametitle{sql.mit.edu} + Though scripts.mit.edu makes use of sql.mit.edu, it's a separate SIPB service with different maintainers. +\begin{itemize} +\item sql.mit.edu provides MySQL databases to scripts users and anyone else +\item SQL data is stored locally, replicated across multiple servers +\item Nightly backups go into AFS +\end{itemize} +\end{frame} + +\subsection{Version control} + +\begin{frame} + \frametitle{SVN and Git hosting} + \begin{itemize} + \item New service (September 2008), not well documented + \item svn://\textit{username}.scripts.mit.edu/ and git://\textit{username}.scripts.mit.edu/ + \item Uses suEXEC to run a svnserve / git-daemon as the user + \item /mit/\textit{username}/Scripts/\{svn,git\} + \item git:// is read-only, so future plans for svn+ssh:// and git+ssh:// + \end{itemize} +\end{frame} Index: branches/fc20-dev/locker/doc/cluedump/slides.tex =================================================================== --- branches/fc20-dev/locker/doc/cluedump/slides.tex (revision 2523) +++ branches/fc20-dev/locker/doc/cluedump/slides.tex (revision 2523) @@ -0,0 +1,42 @@ +\documentclass{beamer} + +\mode +{ \usetheme{Copenhagen} } + +\AtBeginSection[] +{ + \begin{frame} + \frametitle{Outline} + \tableofcontents[currentsection] + \end{frame} +} + +\usepackage{graphicx} + +\title{scripts.mit.edu} +\author{Quentin Smith \and Geoffrey Thomas \\ \texttt{scripts@mit.edu}} +\institute{Student Information Processing Board} +\date{October 28, 2008} + +\begin{document} + +% Title slide - do not change +\begin{frame} + \titlepage +\end{frame} + +\include{contents} +\include{services} + +\include{backend} + +\include{AFS} +\include{suexec} +\include{kerberos} +\include{LDAP} +\include{httpdmods} +\include{LVS} + +\include{closing} + +\end{document} Index: branches/fc20-dev/locker/doc/cluedump/standard-slide-include.sty =================================================================== --- branches/fc20-dev/locker/doc/cluedump/standard-slide-include.sty (revision 2523) +++ branches/fc20-dev/locker/doc/cluedump/standard-slide-include.sty (revision 2523) @@ -0,0 +1,137 @@ +% \pagestyle{empty} % no page numbers + +\addtolength{\topmargin}{-1.25in} +\addtolength{\textheight}{2in} +\frenchspacing % uniform spacing +\hyphenpenalty=10000 % no hyphenation +\rightskip=0pt plus1.4in % add some stretchy glue to right side + +%%% Make LaTeX stuff easier to typeset + +\def\cmd#1{{\tt $\backslash$#1}} + +%\def\example#1{\begin{tabular}{p{0.5\textwidth}p{0.5\textwidth}} #1 &% +%\begin{verbatim} +%#1 +%\end{verbatim} \end{tabular}} + +%\def\example#1{\verb @#1@} + +%%% Set up handling of titles for slides + +\newlength{\titl@wd} %% Length of title +\newlength{\titl@rulewidth} %% Thickness of underlining +\setlength{\titl@rulewidth}{0.15ex} + +\def\titlesize{\large} +\def\textsize{\normalsize} + +\def\mktitle{\slidetitle} + +\def\slidetitle{\@ifstar{\@nocontentstitle}{\@contentstitle}} + +\def\@contentstitle#1{% + \addcontentsline{toc}{slide}{#1}% + \@nocontentstitle{#1}} + +\def\@nocontentstitle#1{{%Makes the title of the slide + \def\th@title{{\titlesize \bf #1}} + \settowidth{\titl@wd}{\th@title} + \flushleft\th@title\hspace{-\titl@wd}%This comment necessary for spacing + \rule[-.3\baselineskip]{\textwidth}{\titl@rulewidth}\hfil\par +}} + +%%% Set up figure and table environments + +%% Some required counters +\newcounter{fig@re} +\setcounter{fig@re}{0} + +\newcounter{t@ble} +\setcounter{t@ble}{0} + +%% The environments themselves +\newenvironment{figure} + {\@floatimitator{fig@re}} + {} + +\newenvironment{table} + {\@floatimitator{t@ble}} + {} + +%% A macro to detect [] stuff +\def\@floatimitator#1{% + \@ifnextchar[% + {\@xfloatimitator{#1}}% [] stuff present + {\@regfloatimitator{#1}}} % call different things depending on whether location specifier is present + +\def\@regfloatimitator#1{% + \stepcounter{#1}% + \def\caption##1{% + {\def\@currentlabel{\csname the#1\endcsname}% + \m@kecaption{\csname #1num\endcsname}{##1}}}} + +\def\@xfloatimitator#1[#2]{\@regfloatimitator{#1}} % just ignore the [] stuff + +%% Caption starter definitions +\def\fig@renum{\fig@rename~\thefig@re} +\def\t@blenum{\t@blename~\thet@ble} + +\def\fig@rename{Figure} +\def\t@blename{Table} + +%% Macro to make caption itself +\long\def\m@kecaption#1#2{% + \vskip\abovecaptionskip + \sbox\@tempboxa{#1: #2}% + \ifdim \wd\@tempboxa >\hsize + #1: #2\par + \else + \global \@minipagefalse + \hb@xt@\hsize{\hfil\box\@tempboxa\hfil}% + \fi + \vskip\belowcaptionskip} + +%% Some more lengths to control caption positioning +\newlength{\abovecaptionskip} +\setlength{\abovecaptionskip}{0pt} + +\newlength{\belowcaptionskip} +\setlength{\belowcaptionskip}{0pt} + + +%%% Label handling + +%% Fix the way labels are handled... +\def\label#1{\@bsphack + \protected@write\@auxout{}% + {\string\newlabel{#1}{{\@currentlabel}{\theslide}}}% + \@esphack} + +%% define the standard label to just be slide +\def\@currentlabel{\theslide} + + +%% table of contents stuff +\def\addcontentsline#1#2#3{% + \addtocontents{#1}{\protect\contentsline{#2}{#3}{\theslide}}} + +\newcommand\tableofcontents{% + \thispagestyle{empty}% + \addtocounter{slide}{-1}% + \slidetitle*{\contentsname + \@mkboth{% + \MakeUppercase\contentsname}{\MakeUppercase\contentsname}}% + \begingroup + \tiny + \@starttoc{toc}% + \endgroup + } + +\newcommand*\l@slide{\@dottedtocline{0}{1.5em}{2.3em}} + +\newcommand\contentsname{Contents} +\newcommand\@pnumwidth{1.55em} +\newcommand\@tocrmarg{2.55em} +\newcommand\@dotsep{4.5} +\setcounter{tocdepth}{0} Index: branches/fc20-dev/locker/doc/cluedump/suexec.tex =================================================================== --- branches/fc20-dev/locker/doc/cluedump/suexec.tex (revision 2523) +++ branches/fc20-dev/locker/doc/cluedump/suexec.tex (revision 2523) @@ -0,0 +1,96 @@ +\subsection{suEXEC} + +\begin{frame} + \frametitle{Serving static content} + \begin{itemize} + \item The \texttt{apache} user does not have permission to read the + user's files directly. + \item Both static and dynamic content is served through suEXEC. + \end{itemize} +\end{frame} + +\begin{frame}[fragile,t] + \begin{enumerate} + \item \texttt{/etc/httpd/conf.d/execsys.conf} is configured to serve + static content with the \texttt{cgi-script} handler. + \end{enumerate} +\begin{footnotesize} +\begin{semiverbatim} + + SetHandler cgi-script + Options +ExecCGI + + + SetHandler cgi-script + Options +ExecCGI + +\ldots + + SetHandler cgi-script + Options +ExecCGI + + + SetHandler cgi-script + Options +ExecCGI + +\ldots +\end{semiverbatim} +\end{footnotesize} +\end{frame} + +\begin{frame}[fragile,t] + \begin{enumerate} + \addtocounter{enumi}{1} + \item \texttt{openafs/src/afs/VNOPS/afs\_vnop\_access.c} is modified + to mark \emph{all} files as executable (!). + \end{enumerate} +\begin{footnotesize} +\begin{semiverbatim} + int + afs_access(OSI_VC_DECL(avc), register afs_int32 amode, + struct AFS_UCRED *acred) + \{ + register afs_int32 code; + struct vrequest treq; + struct afs_fakestat_state fakestate; + OSI_VC_CONVERT(avc); + + AFS_STATCNT(afs_access); ++ amode = amode & ~VEXEC; + afs_Trace3(afs_iclSetp, CM_TRACE_ACCESS, ICL_TYPE_POINTER, avc, + ICL_TYPE_INT32, amode, ICL_TYPE_OFFSET, + ICL_HANDLE_OFFSET(avc->m.Length)); + \ldots + \} +\end{semiverbatim} +\end{footnotesize} +\end{frame} + +\begin{frame}[fragile,t] + \begin{enumerate} + \addtocounter{enumi}{2} + \item \texttt{httpd/support/suexec.c} is modified to dispatch static + content to \texttt{/usr/local/bin/static-cat}. + \end{enumerate} +\begin{footnotesize} +\begin{semiverbatim} ++#define STATIC_CAT_PATH "/usr/local/bin/static-cat" ++static const char *static_extensions[] = \{ ++ "html", ++ "css", ++ \ldots ++\} ++ + int main(int argc, char *argv[]) + \{ + \ldots ++ if (is_static_extension(cmd)) \{ ++ argv[2] = STATIC_CAT_PATH; ++ execv(STATIC_CAT_PATH, &argv[2]); ++ log_err("(%d)%s: static_cat exec failed (%s)\\n", errno, ++ strerror(errno), argv[2]); ++ exit(255); ++ \} +\end{semiverbatim} +\end{footnotesize} +\end{frame} Index: branches/fc20-dev/locker/doc/object-identifiers =================================================================== --- branches/fc20-dev/locker/doc/object-identifiers (revision 2523) +++ branches/fc20-dev/locker/doc/object-identifiers (revision 2523) @@ -0,0 +1,1 @@ +link /afs/sipb.mit.edu/admin/text/object-identifiers Index: branches/fc20-dev/locker/doc/scripts-admin-use-policy =================================================================== --- branches/fc20-dev/locker/doc/scripts-admin-use-policy (revision 2523) +++ branches/fc20-dev/locker/doc/scripts-admin-use-policy (revision 2523) @@ -0,0 +1,49 @@ + 2008-03-15 + amended 2008-08-05 +Policy on the Use of scripts.mit.edu Administrative Rights + +Users of scripts.mit.edu have a reasonable expectation that the data +and code they store on our servers, and in sections of their locker +accessible only by our servers, will not be improperly accessed or +modified by anyone else, including by scripts.mit.edu maintainers. To +fulfill this expectation, we define a policy governing the +maintainers’ use of special permissions and credentials held by our +servers. This includes any administrative access to the scripts +servers, any use of private keys stored on the servers, and any use of +scripts-specific permissions granted on locker directories. + +Such use of administrative rights shall only be permitted under any of +the following circumstances. + +* Maintenance of the scripts.mit.edu service itself that is unrelated + to private user data. + +* Any access that is explicitly authorized by the owners of the data + in question. + +* Handling a user support request that cannot be satisfactorily answered + without resorting to using administrative rights. This access should + be restricted to only those files and resources that are strictly + necessary to fully answer the request. + +* Performing upgrades to autoinstalled software, using permissions + granted to the system:scripts-security-upd group. This group is + normally empty, but the root instances of scripts maintainers will + be added when needed to perform upgrades, at the discretion of the + architect. + +* Modifications that are necessary for server security or reliability. + In this case, any modifications should be clearly marked and the + user should be contacted. + +* Ensuring that updates or planned updates to the scripts.mit.edu + service do not break existing user deployments. In this case, any + modifications should be clearly marked and the user should be + contacted. + +[The third clause formerly read +* Handling a user support request that can reasonably be considered an + implicit authorization for that use. In this case, whenever + possible, any modifications should be reverted and the user should + be told how to make these modifications themselves. +and was changed in August 2008.] Index: branches/fc20-dev/locker/doc/scripts-code-review =================================================================== --- branches/fc20-dev/locker/doc/scripts-code-review (revision 2523) +++ branches/fc20-dev/locker/doc/scripts-code-review (revision 2523) @@ -0,0 +1,25 @@ +Scripts has a policy of formal code review on Zephyr. Commit messages +are zephyred to -c scripts -i r[number], with the actual diff sent to +-c scripts-auto -i commits. Commits are also emailed to +scripts-commits@mit.edu. All commits should be reviewed by another +scripts-team member before they are deployed. + +The following designations are in use, with the following meanings: +"+1": I have reviewed and approve of this commit. + +"+0": I've glanced at the commit and it seems okay. + +"-0": I have mild issues with this commit, but am fine with it going + in as-is and no further action being taken. + +"-1": I do not approve of this commit, it needs more work. This + designation must include an explanation of the objection, and + will likely result in further action being taken either by the + committer or the reviewer to improve the commit. + +"-1, revert": I do not approve of this commit, and believe it to be + actively harmful to the project, such as introducing a security + hole. This designation must include an explanation of the + objection. The committer should revert the commit. + +Reviewers should strive to give non-zero reviews whenever possible. Index: branches/fc20-dev/locker/doc/scripts-decision-policy =================================================================== --- branches/fc20-dev/locker/doc/scripts-decision-policy (revision 2523) +++ branches/fc20-dev/locker/doc/scripts-decision-policy (revision 2523) @@ -0,0 +1,131 @@ + 2007-07-07 +The Decision-Making Policy of the scripts.mit.edu Project: + +We, the creators of the scripts.mit.edu infrastructure, wish to define a +policy for how decisions of the scripts.mit.edu project will be reached in +order to avoid confusion on this subject among future contributors to the +project. We particularly want to avoid a situation in which the +leadership of the project is unclear after we leave MIT. + +In general, we believe that all contributors to the project should have a +say in how the service is run in approximate proportion to their +contributions. We furthermore believe that strong agreement among the +project's principal contributors is highly important to the project's +future, and so, whenever possible, the project's principal contributors +should reach near-unanimous agreement about how the project should +proceed. Ultimately, the decisions of a project of this nature need to be +made by the people who are making the project happen. + +Unfortunately, reaching unanimous agreement among all of the contributors +to the project might not always be possible. This document establishes +two leadership positions for the scripts.mit.edu project in order to +entrust decision-making authority to specific individuals. These leaders +are ultimately entrusted with the project, although they are expected to +take significant pause before using their authority to end a disagreement +before consensus of the principal contributors has been reached. These +leadership positions are based in part on the roles of "producer" and +"director" described in Frederick P. Brooks' _The Mythical Man-Month_. + +The "scripts team leader" is an MIT student who: +- "assembles the team, divides the work, and establishes the schedule" +- "acquires and keeps on acquiring the necessary resources" +- "establishes the pattern of communication and reporting within the team" +- "ensures that the schedule is met, shifting resources and organization + in order to respond to changing circumstances" + +The team leader is responsible for ensuring that the project continues to +make regular progress. The team leader is entrusted with arbitrating +decisions regarding the organization of the scripts team and the focus of +its ongoing development efforts. For example, the team leader may remove +individuals from the project who are deemed to be having an overall +negative influence on the project. + +The "scripts architect" is an MIT student who: +- "provides unity and conceptual integrity to the whole design" +- "serves as a limit to system complexity" +- "invents solutions for [large-scale technical problems] or shifts the + system design as required" + +The architect is responsible for ensuring the technical quality of the +scripts.mit.edu service. The architect is entrusted with arbitrating +decisions regarding the scope, design, and operation of the service. As +the guardian of the technical integrity of the service, the architect may +arbitrate all decisions regarding the project's production hardware and +software. + +Both positions may select their own replacement, and, in the case of a +vacancy, either position may select a replacement for the other position. +Before an individual assumes either position as a replacement, that +individual should be confirmed for that position by the SIPB Executive +Committee. A single individual may hold both positions simultaneously if +every individual who has significantly contributed to the project within +the last one calendar year agrees. Any objections must occur before the +Executive Committee has confirmed the appointment. + +The creator of the scripts.mit.edu project, Jeff Arnold, will serve as the +first team leader and architect. + +Any part of the scripts.mit.edu decision-making policy may be modified as +necessary by agreement between the scripts team leader and the scripts +architect. When changing the scripts.mit.edu decision-making policy, as +with any major decision, near-unanimous agreement among the project's +principal contributors should ideally be reached. + +The scripts.mit.edu project is affiliated with SIPB, and while the project +remains affiliated with SIPB, the project will follow appropriate SIPB +procedures for projects. + +This policy should be distributed to contributors to the project so that +they may decide not to contribute if they are dissatisfied with it. + + + + + + + ____________________________________ + jbarnold + + + + + + + ____________________________________ + presbrey + + + + + + + ____________________________________ + hartmans + + +As contributors to the scripts.mit.edu project, we have contributed to +the creation of this written decision-making policy and we fully support it. + + + + + + + ____________________________________ + tabbott + + + + + + + ____________________________________ + andersk + + + + + + + ____________________________________ + geofft Index: branches/fc20-dev/locker/doc/tickets/cnames.txt =================================================================== --- branches/fc20-dev/locker/doc/tickets/cnames.txt (revision 2523) +++ branches/fc20-dev/locker/doc/tickets/cnames.txt (revision 2523) @@ -0,0 +1,87 @@ +HANDLING CNAME REQUESTS + +When someone e-mails scripts.mit.edu asking for a foo.mit.edu hostname: + +1. Check that the hostname is not currently in use. The commands + stella foo.mit.edu + athrun ops qy ghal foo.mit.edu \* + should both say the name is not in use. (The latter checks for aliases of + deleted or otherwise inactive hostnames that stella ignores.) + + If the name is currently an alias of a name they own, make sure to forward + to jweiss the permission to move that name around. + + If the name is the primary name of a machine they own, ask them what they + would like to rename the machine to, and make it clear that they'll need to + have another name associated with that IP address. Or (especially if the + machine doesn't ping) ask them to confirm they're no longer using that IP + address. If they're totally confused and keep insisting they want scripts + to serve that name, go ahead and tell them you'll rename the current foo to + foo-old. + + If the name belongs to a deleted host on a dorm network, e-mail rccsuper to + reap it; they should do so quickly. If it belongs to an FSILG, e-mail + ht-$ILG-acl (ht-et-acl, ht-pika-acl, etc.) and ask nicely. If it belongs + to an academic network, they're not getting even deleted names back unless + they can negotiate with the current owner of the name. + +2. Check that they're requesting a scripts.mit.edu path that they control + (preferably, they'll give you a locker.scripts.mit.edu/something URL). If + they want a web.mit.edu path, you'll need to tell them to set up a redirect + according to http://scripts.mit.edu/faq/63/ in a directory in their + web_scripts, and ask them to tell us the directory. This doesn't block + requesting the hostname. + + If they want something more outlandish, make sure they're not confused + before proceeding. + + Confirm that they're signed up for scripts. http://locker.scripts.mit.edu/ + should give something that isn't the "Account unknown" page. + +3. E-mail jweiss. + * Open the ticket in RT + * Click 'Comment' to the right of the body of the e-mail they sent + * CC: jweiss@mit.edu (Don't use "To:", there's a bug) + * Write something nice, preferably including the stella command line. + I typically use + Subject: scripts CNAME request: foo.mit.edu + + At your convenience, please make foo.mit.edu an alias of scripts-vhosts.mit.edu. + + stella scripts-vhosts -a foo + (or) + At your convenience, please move the alias foo.mit.edu from bar.mit.edu to + scripts-vhosts.mit.edu. + + stella bar -d foo + stella scripts-vhosts -a foo + (or) + At your convenience, please rename the current host foo.mit.edu to + foo-old.mit.edu and mark it as deleted, and make foo.mit.edu an + alias of scripts-vhosts.mit.edu. + + stella foo -S 3 -R foo-old + stella scripts-vhosts -a foo + (or) + If the request below is sufficient authorization, please remove.... + + * Set Status => Waiting and Blocking On => Moira + + Occasionally jweiss is on vacation; generally he sets an auto-responder, + so you can just try emailing him and hoping, or ask him if he's around. If + not, see if zacheiss or cfox or computing-help will handle the requests. + (zacheiss has been willing to do them in the past.) + +4. Reply to the requestor (from either RT or your e-mail client), with + something like "We've forwarded the hostname request to IS&T; it should take + effect in 2-3 business days." + +5. After the name updates (jweiss replies, and DNS updates - which you can + check on -i dns), ask someone with root access to run + + vhostadd foo.mit.edu + +6. Reply to the requestor again, and help them with stuff like MediaWiki URLs + or RewriteRules if they're having trouble. + +--geofft with minor edits by adehnert, last updated 2009-12-01 Index: branches/fc20-dev/locker/doc/tickets/rt.txt =================================================================== --- branches/fc20-dev/locker/doc/tickets/rt.txt (revision 2523) +++ branches/fc20-dev/locker/doc/tickets/rt.txt (revision 2523) @@ -0,0 +1,58 @@ +RT TRICKS + +To edit stuff like ticket status, click "Basic" in the left. + +Note the multiple ways to search for tickets: you can click "All +{new,open,waiting} Scripts Tickets" on the home page in the center, or +"Scripts" on the right in the list of queues. + +You should take a look in "Preferences" at the left. Make sure "Notify +yourself of own updates" is on. You can also set the "Default Working +Queue" to Scripts, and give yourself a signature referring to +scripts@mit.edu. + +Another useful option here is to set an RT password for your account, +so you don't need certs to log in (though it sometimes will keep +asking you for your password on each page load if you don't have +certs) and so you can use the BarnOwl RT module or the zephyrbot. +The BarnOwl RT module is the preferred option. To install it, run + + mkdir -p ~/.owl/modules + cd ~/.owl/modules + ln -s /mit/snippets/rt/BarnOwl RT + +Then in BarnOwl run ":reload-module RT". The RT module adds the :rt +command to barnowl. Run ":rt help" for more information; useful +shortcuts are ":rt r" to set the status to resolved and ":rt d" to set +the status to deleted. The zephyrbot will take commands to -c scripts +-i [ticket number] of the form /set status=resolved or /set +owner=geofft. Talk to ezyang to be added to the zephyrbot. + +Note that in the event that Edward's account is compromised, it is +possible for an attacker to use this password do manipulate tickets in +*any* queue you have bits on, not just the Scripts one. + +The RT bot will post ticket notifications as -c scripts -i nnn. If +you are responding to a ticket, it is conventional to post "lock" to +the appropriate instance, so others know not to pre-empt you. You +should post "unlock" once you are done handling the ticket. + +You can also place these commands on a line by themselves inside +e-mail; they will be acted upon and removed before the e-mail gets +sent back out. + +If you're adding a *comment* (such as when you're forwarding a cname +request on to IS&T), don't use the 'To:' field, because it'll be +clobbered by our RT scrips and the mail won't actually go to the +destination you added. Instead, you should use the 'Cc:' field. +Similarly, if you're adding *correspondence*, the 'Cc:' field will be +clobbered and you need to use the 'To:' field. + +Don't CC other RT queues, it doesn't work. If you really need to, use +your e-mail client to forward it and remove the [help.mit.edu #nnn] +tag. + +E-mail to scripts-comment that carries a [help.mit.edu #nnn] tag will +be included in the ticket history for the scripts team to see, but +will not be sent to the user. You can use this for asking "Help, what +do I do here?" Index: branches/fc20-dev/locker/etc/known_hosts =================================================================== --- branches/fc20-dev/locker/etc/known_hosts (revision 2523) +++ branches/fc20-dev/locker/etc/known_hosts (revision 2523) @@ -0,0 +1,1 @@ +scripts,scripts.mit.edu,scripts-vhosts,scripts-vhosts.mit.edu,scripts-test,scripts-test.mit.edu,b-k,b-k.mit.edu,bees-knees,bees-knees.mit.edu,b-m,b-m.mit.edu,better-mousetrap,better-mousetrap.mit.edu,b-b,b-b.mit.edu,busy-beaver,busy-beaver.mit.edu,c-w,c-w.mit.edu,cats-whiskers,cats-whiskers.mit.edu,g-e,g-e.mit.edu,golden-egg,golden-egg.mit.edu,l-s,l-s.mit.edu,lucky-star,lucky-star.mit.edu,m-c,m-c.mit.edu,miracle-cure,miracle-cure.mit.edu,o-f,o-f.mit.edu,old-faithful,old-faithful.mit.edu,p-b,p-b.mit.edu,pancake-bunny,pancake-bunny.mit.edu,r-m,r-m.mit.edu,real-mccoy,real-mccoy.mit.edu,s-a,s-a.mit.edu,shining-armor,shining-armor.mit.edu,w-e,w-e.mit.edu,whole-enchilada,whole-enchilada.mit.edu,scripts1,scripts1.mit.edu,scripts2,scripts2.mit.edu,scripts3,scripts3.mit.edu,scripts4,scripts4.mit.edu,scripts5,scripts5.mit.edu,scripts6,scripts6.mit.edu,scripts7,scripts7.mit.edu,scripts8,scripts8.mit.edu,scripts9,scripts9.mit.edu,scripts10,scripts10.mit.edu,scripts11,scripts11.mit.edu,scripts12,scripts12.mit.edu,18.181.0.43,18.181.0.46,18.181.0.57,18.181.0.53,18.181.0.167,18.181.0.228,18.181.0.236,18.181.0.237,18.181.0.234,18.181.0.235,18.181.0.135,18.181.0.141,18.181.0.203,18.181.0.204,18.181.0.229 ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuEpkEgaIgjK7F1gV81lLSYTwSqIZX/9IJs37VaJCsJFv3D86uuJSdfI3Y94fPn2OH6AxfdaqGNksVdi27mKQfzvCB4ogjQgxmM391MIDLd+izZDY0YvCb4DqJLMJUpX49cNUMkj+/rJg1O0K2w/lb8DGr7wdoLSPKCUJNJv5WMMDxpFL253lPELsmnds4T+R6LpTt6W9+FalHl84me51sEjV9PbmhcTaNwuoJStAjhrKPfgHHDIKNyCUvaVkoHPXEsdzz00yY7i57djyZlzPV/jM7LKar+Xw2LB0Z3098IQcdbD8zmz2DdakPTlShxavNPC6kZDZ3WVqziC+bszaSQ== Index: branches/fc20-dev/locker/sbin/build-update =================================================================== --- branches/fc20-dev/locker/sbin/build-update (revision 2523) +++ branches/fc20-dev/locker/sbin/build-update (revision 2523) @@ -0,0 +1,155 @@ +#!/usr/bin/env perl + +use File::Spec::Functions qw(:ALL); +use File::Copy; +use File::Path; +use Data::Dumper; +use Getopt::Long; +use Archive::Tar; +use Cwd; +use strict; +use warnings; + +my $scriptsdev = ""; + +GetOptions( +#"redo-delete" => \$redodelete, +# "redo-add" => \$redoadd, +# "redo-replace" => \$redoreplace, +# "redo-diff" => \$redodiff, +# "redo-all" => sub {$redodelete = $redoadd = $redoreplace = $redodiff = 1;}, + "dev" => sub {$scriptsdev = "dev";}, + ); + +if (@ARGV < 3) { + print STDERR "Usage: $0 package oldversion newversion\n"; + exit(1); +} + +my ($package, $oldversion, $newversion) = @ARGV; +my ($old, $new, $updatename) = ($package.'-'.$oldversion, $package.'-'.$newversion, $package.'-'.$oldversion.'-to-'.$newversion); + +my $proposaldir = $updatename.".proposal"; +my $outdir = catdir("/mit/scripts/deploy$scriptsdev/updates/", $updatename); + +(-d $proposaldir) or die "Can't find $proposaldir, did you run propose-update?\n"; +((! -e $outdir) && mkdir($outdir)) or die "mkdir($outdir) failed: $!"; + +my $olddir = catdir($proposaldir,$old); +my $newdir = catdir($proposaldir,$new); +(-d $olddir && -d $newdir) or die "Packages not unpacked?\n"; + +sub readFileList($) { + my ($list) = @_; + open(LIST, $list) or die "Can't read $list: $!\n"; + my @files = map { chomp; s|$newdir\/?||g; [split(' ', $_, 2)] } ; + close(LIST); + return @files; +} + +my @todelete = readFileList(catfile($proposaldir, "files.delete")); +my @toadd = readFileList(catfile($proposaldir, "files.add")); +my @toreplace = readFileList(catfile($proposaldir, "files.replace")); + +open(CHECKMD5, ">", catfile($outdir, "check.md5")) or die "Can't open check.md5: $!"; +print CHECKMD5 map { join(" ", @{$_})."\n" } @todelete, @toreplace; +close(CHECKMD5); + +open(ABSENT, ">", catfile($outdir, "oldfiles.absent")) or die "Can't open oldfiles.absent: $!"; +print ABSENT map { $_->[1]."\n" } @toadd; +close(ABSENT); + +open(DELETE, ">", catfile($outdir, "files.delete")) or die "Can't open files.delete: $!"; +print DELETE map { $_->[1]."\n" } @todelete, @toreplace; +close(DELETE); + +my $outfiletarball = catfile($outdir, "newfiles.tar.gz"); +my $cwd = getcwd(); + +chdir($newdir); +my $tar = Archive::Tar->new; +$tar->add_files(map { $_->[1] } @toadd, @toreplace); +$tar->write($outfiletarball, 1); +chdir($cwd) or die "Couldn't chdir back to $cwd: $!\n"; + +if (0) { + my $outfiledir = catdir($outdir, "newfiles"); + (-d $outfiledir || mkdir($outfiledir)) or die "Can't mkdir($outfiledir)\n"; + + foreach my $file (@toadd, @toreplace) { + my $filename = $file->[1]; + my $src = catfile($newdir, $filename); + my $dest = catfile($outfiledir, $filename); + my (undef, $dir, undef) = splitpath($dest); + mkpath($dir); + copy($src, $dest) or die "Couldn't copy $filename from $src to $dest: $!"; + } +} + +copy(catfile($proposaldir, "update.diff"), catfile($outdir, "update.diff")) or die "Couldn't copy update.diff: $!"; +if (-d catdir($proposaldir, "extra")) { + system('cp', '-r', catdir($proposaldir, "extra"), $outdir) && die "Couldn't copy extra: $!"; +} + +open (SCRIPT, ">", catfile($outdir, "update")) or die "Couldn't write update: $!"; +printf SCRIPT <<'EOF', catfile("/afs/athena.mit.edu/contrib/scripts/deploy/", $old), catfile("/afs/athena.mit.edu/contrib/scripts/deploy/", $new); +#!/bin/bash +set -e -o noclobber + +die () { echo "[$PWD] die: $1" >&2; rm .scripts-security-upd-lock; exit 1; } +aiee () { echo "[$PWD] AIEE: $1" >&2; exit 2; } + +patchdir=$(dirname "$0") +lockername="${1:-$USER}" + +>.scripts-security-upd-lock || die "error: Unable to obtain update lock." + +packages=$(tail -1 .scripts-version) + +echo "[$PWD] begin dry run" + +found="" +newpackages="" +for package in $packages; do + if [ "$package" = "%s" ]; then + found="yes" + newpackages="$newpackages%s " + else + newpackages="$newpackages$package " + fi +done + +[ "$found" = "yes" ] || die "error: Correct version not found." + +if [ -e "$patchdir/extra/prepatch.sh" ]; then + . "$patchdir/extra/prepatch.sh" || die "error: prepatch failed" +fi + +[ -s "$patchdir/oldfiles.absent" ] && (xargs -n1 test ! -e <"$patchdir/oldfiles.absent" || die "error: Conflicting files exist." ) +[ -s "$patchdir/check.md5" ] && (md5sum -c "$patchdir/check.md5" || die "error: MD5 check failed.") +patch -stN --dry-run --no-backup-if-mismatch -p2 <"$patchdir/update.diff" || die "error: Patch dry run failed." + +echo "[$PWD] dry run OK, applying update" + +mv .scripts-version .scripts-version.old || aiee "Failed to back up scripts-version" +patch -stN --no-backup-if-mismatch -p2 <"$patchdir/update.diff" || aiee "patch did not apply" +xargs rm -f <"$patchdir/files.delete" || aiee "Failed to remove files" +/afs/athena.mit.edu/contrib/scripts/bin/gtar -xzf "$patchdir/newfiles.tar.gz" || aiee "Failed to extract new files" +[ -e "$patchdir/extra/postpatch.sh" ] && . "$patchdir/extra/postpatch.sh" +( + cat .scripts-version.old + echo + date "+%%F %%T %%z" + echo "$USER@$(hostname)" + echo "$patchdir" + echo "$newpackages" +) >.scripts-version +rm -f .scripts-version.old || aiee "Failed to remove .scripts-version.old" + +rm -f .scripts-security-upd-lock || aiee "Failed to remove .scripts-security-upd-lock" + +echo "[$PWD] done" + +exit 0 +EOF +chmod 0755, catfile($outdir, "update"); Index: branches/fc20-dev/locker/sbin/check-ldap-cert.pl =================================================================== --- branches/fc20-dev/locker/sbin/check-ldap-cert.pl (revision 2523) +++ branches/fc20-dev/locker/sbin/check-ldap-cert.pl (revision 2523) @@ -0,0 +1,32 @@ +#!/usr/bin/perl + +use strict; +use File::Basename; +use Date::Parse; +use Sys::Hostname; + +my @servers = qw(localhost); + +my $hostname = hostname(); + +my $now = time(); + +my $dir = dirname($0); + +our $verbose = 0; +$verbose = 1 if ($ARGV[0] eq "-v"); + +use constant WARNING => 60*60*24*14; # Warn if a cert is expiring within 14 days + +foreach my $server (@servers) { + open(X509, "-|", "$dir/ssl-get-endtime", "$server:636") or die "Couldn't invoke ssl-get-endtime: $!"; + chomp(my $exp = ); + close(X509); + $exp =~ s/^notAfter=// or warn "Cert appears broken: $server"; + + my $time = str2time($exp); + + if ($verbose || ($time - $now) <= WARNING) { + printf "$hostname: Certificate expiring in %.2f days: %s\n", (($time - $now) / (60.0*60*24)), $server; + } +} Index: branches/fc20-dev/locker/sbin/check-users =================================================================== --- branches/fc20-dev/locker/sbin/check-users (revision 2523) +++ branches/fc20-dev/locker/sbin/check-users (revision 2523) @@ -0,0 +1,79 @@ +#!/bin/bash +set -e + +err() { + echo "$@" +} + +if [ -z "$1" ]; then + filter="objectClass=posixAccount" +else + filter= + for user; do + filter="$filter(uid=$user)" + done + filter="(&(objectClass=posixAccount)(|$filter))" +fi + +unset "${!l_@}" +while read attr value; do + if [ -n "$attr" ]; then + declare "l_${attr%:}=$value" + continue + fi + + read f_type f_data < <(hesinfo "$l_uid" filsys) || : + if [ -z "$f_type" ]; then + err "$l_uid" "no_hesiod" + elif [ "$f_type" = "ERR" ]; then + err "$l_uid" "hesiod_err ERR $f_data" + elif [ "$f_type" = "AFS" ]; then + read f_path f_perm f_link z \ + < <(echo "$f_data") + [ "$l_homeDirectory" = "$f_path" ] || \ + err "$l_uid" "hesiod_path $f_path" + else + err "$l_uid" "wrong_hesiod $f_type" + fi + + IFS=/ read p_empty p_top p_cell p \ + < <(echo "$l_homeDirectory") + [ -z "${p_empty}" ] || \ + err "$l_uid" "relative_home $l_homeDirectory" + [ "${p_top}" = "afs" ] || \ + err "$l_uid" "not_afs $l_homeDirectory" + + read v_vname v_vol v \ + < <(vos exa -noauth "$l_uidNumber" -cell "$p_cell" 2>/dev/null) || : + [ "$v_vol" = "$l_uidNumber" ] || + err "$l_uid" "no_vol $l_uidNumber" + + if ! [ -d "$l_homeDirectory" ]; then + if ! [ -e "$l_homeDirectory" ]; then + err "$l_uid" "deleted $l_homeDirectory" + else + err "$l_uid" "not_dir $l_homeDirectory" + fi + else + read c c_path c c c c_cell \ + < <(fs whichcell "$l_homeDirectory" 2>/dev/null) || : + [ "$c_path" = "$l_homeDirectory" ] || \ + err "$l_uid" "no_cell $l_homeDirectory" + [ "$c_cell" = "'$p_cell'" ] || \ + err "$l_uid" "wrong_cell $l_homeDirectory" + + read m_path m m m m m m m_vname \ + < <(fs lsmount "$l_homeDirectory" 2>/dev/null) || : + [ "$m_path" = "'$l_homeDirectory'" ] || \ + err "$l_uid" "no_mount $l_homeDirectory" + [ "$m_vname" = "'#$v_vname'" ] || [ "$m_vname" = "'%$v_vname'" ] || \ + err "$l_uid" "wrong_mount $m_vname ($l_uidNumber = $v_vname)" + fi + + unset "${!l_@}" +done < <( + ldapsearch -LLL -x -D 'cn=Directory Manager' -y /etc/signup-ldap-pw \ + -b ou=People,dc=scripts,dc=mit,dc=edu "$filter" \ + uid uidNumber homeDirectory loginShell | \ + perl -0pe 's/\n //g;' + ) Index: branches/fc20-dev/locker/sbin/commit-email.pl =================================================================== --- branches/fc20-dev/locker/sbin/commit-email.pl (revision 2523) +++ branches/fc20-dev/locker/sbin/commit-email.pl (revision 2523) @@ -0,0 +1,840 @@ +#!/usr/bin/env perl + +# ==================================================================== +# This script is deprecated. The Subversion developers recommend +# using mailer.py for post-commit and post-revprop change +# notifications. If you wish to improve or add features to a +# post-commit notification script, please do that work on mailer.py. +# See http://svn.collab.net/repos/svn/trunk/tools/hook-scripts/mailer . +# ==================================================================== + +# ==================================================================== +# commit-email.pl: send a notification email describing either a +# commit or a revprop-change action on a Subversion repository. +# +# For usage, see the usage subroutine or run the script with no +# command line arguments. +# +# This script requires Subversion 1.2.0 or later. +# +# $HeadURL: http://svn.collab.net/repos/svn/trunk/contrib/hook-scripts/commit-email.pl.in $ +# $LastChangedDate: 2009-05-12 13:25:35 -0400 (Tue, 12 May 2009) $ +# $LastChangedBy: blair $ +# $LastChangedRevision: 37715 $ +# +# ==================================================================== +# Copyright (c) 2000-2006 CollabNet. All rights reserved. +# +# This software is licensed as described in the file COPYING, which +# you should have received as part of this distribution. The terms +# are also available at http://subversion.tigris.org/license-1.html. +# If newer versions of this license are posted there, you may use a +# newer version instead, at your option. +# +# This software consists of voluntary contributions made by many +# individuals. For exact contribution history, see the revision +# history and logs, available at http://subversion.tigris.org/. +# ==================================================================== + +# Turn on warnings the best way depending on the Perl version. +BEGIN { + if ( $] >= 5.006_000) + { require warnings; import warnings; } + else + { $^W = 1; } +} + +use strict; +use Carp; +use POSIX qw(strftime); +my ($sendmail, $smtp_server); + +###################################################################### +# Configuration section. + +$ENV{'LC_ALL'} = 'en_US.UTF-8'; + +# Sendmail path, or SMTP server address. +# You should define exactly one of these two configuration variables, +# leaving the other commented out, to select which method of sending +# email should be used. +# Using --stdout on the command line overrides both. +$sendmail = "/usr/sbin/sendmail"; +#$smtp_server = "127.0.0.1"; + +# Svnlook path. +my $svnlook = "/usr/bin/svnlook"; + +# By default, when a file is deleted from the repository, svnlook diff +# prints the entire contents of the file. If you want to save space +# in the log and email messages by not printing the file, then set +# $no_diff_deleted to 1. +my $no_diff_deleted = 0; +# By default, when a file is added to the repository, svnlook diff +# prints the entire contents of the file. If you want to save space +# in the log and email messages by not printing the file, then set +# $no_diff_added to 1. +my $no_diff_added = 0; + +# End of Configuration section. +###################################################################### + +# Check that the required programs exist, and the email sending method +# configuration is sane, to ensure that the administrator has set up +# the script properly. +{ + my $ok = 1; + foreach my $program ($sendmail, $svnlook) + { + next if not defined $program; + if (-e $program) + { + unless (-x $program) + { + warn "$0: required program `$program' is not executable, ", + "edit $0.\n"; + $ok = 0; + } + } + else + { + warn "$0: required program `$program' does not exist, edit $0.\n"; + $ok = 0; + } + } + if (not (defined $sendmail xor defined $smtp_server)) + { + warn "$0: exactly one of \$sendmail or \$smtp_server must be ", + "set, edit $0.\n"; + $ok = 0; + } + exit 1 unless $ok; +} + +require Net::SMTP if defined $smtp_server; + +###################################################################### +# Initial setup/command-line handling. + +# Each value in this array holds a hash reference which contains the +# associated email information for one project. Start with an +# implicit rule that matches all paths. +my @project_settings_list = (&new_project); + +# Process the command line arguments till there are none left. +# In commit mode: The first two arguments that are not used by a command line +# option are the repository path and the revision number. +# In revprop-change mode: The first four arguments that are not used by a +# command line option are the repository path, the revision number, the +# author, and the property name. This script has no support for the fifth +# argument (action) added to the post-revprop-change hook in Subversion +# 1.2.0 yet - patches welcome! +my $repos; +my $rev; +my $author; +my $propname; + +my $mode = 'commit'; +my $date; +my $diff_file; + +# Use the reference to the first project to populate. +my $current_project = $project_settings_list[0]; + +# This hash matches the command line option to the hash key in the +# project. If a key exists but has a false value (''), then the +# command line option is allowed but requires special handling. +my %opt_to_hash_key = ('--from' => 'from_address', + '--revprop-change' => '', + '-d' => '', + '-h' => 'hostname', + '-l' => 'log_file', + '-m' => '', + '-r' => 'reply_to', + '-s' => 'subject_prefix', + '--summary' => '', + '--diff' => '', + '--stdout' => ''); + +while (@ARGV) + { + my $arg = shift @ARGV; + if ($arg =~ /^-/) + { + my $hash_key = $opt_to_hash_key{$arg}; + unless (defined $hash_key) + { + die "$0: command line option `$arg' is not recognized.\n"; + } + + my $value; + if ($arg ne '--revprop-change' and $arg ne '--stdout' and $arg ne '--summary') + { + unless (@ARGV) + { + die "$0: command line option `$arg' is missing a value.\n"; + } + $value = shift @ARGV; + } + + if ($hash_key) + { + $current_project->{$hash_key} = $value; + } + else + { + if ($arg eq '-m') + { + $current_project = &new_project; + $current_project->{match_regex} = $value; + push(@project_settings_list, $current_project); + } + elsif ($arg eq '-d') + { + if ($mode ne 'revprop-change') + { + die "$0: `-d' is valid only when used after" + . " `--revprop-change'.\n"; + } + if ($diff_file) + { + die "$0: command line option `$arg'" + . " can only be used once.\n"; + } + $diff_file = $value; + } + elsif ($arg eq '--revprop-change') + { + if (defined $repos) + { + die "$0: `--revprop-change' must be specified before" + . " the first non-option argument.\n"; + } + $mode = 'revprop-change'; + } + elsif ($arg eq '--diff') + { + $current_project->{show_diff} = parse_boolean($value); + } + elsif ($arg eq '--stdout') + { + $current_project->{stdout} = 1; + } + elsif ($arg eq '--summary') + { + $current_project->{summary} = 1; + } + else + { + die "$0: internal error:" + . " should not be handling `$arg' here.\n"; + } + } + } + else + { + if (! defined $repos) + { + $repos = $arg; + } + elsif (! defined $rev) + { + $rev = $arg; + } + elsif (! defined $author && $mode eq 'revprop-change') + { + $author = $arg; + } + elsif (! defined $propname && $mode eq 'revprop-change') + { + $propname = $arg; + } + else + { + push(@{$current_project->{email_addresses}}, $arg); + } + } + } + +if ($mode eq 'commit') + { + &usage("$0: too few arguments.") unless defined $rev; + } +elsif ($mode eq 'revprop-change') + { + &usage("$0: too few arguments.") unless defined $propname; + } + +# Check the validity of the command line arguments. Check that the +# revision is an integer greater than 0 and that the repository +# directory exists. +unless ($rev =~ /^\d+/ and $rev > 0) + { + &usage("$0: revision number `$rev' must be an integer > 0."); + } +unless (-e $repos) + { + &usage("$0: repos directory `$repos' does not exist."); + } +unless (-d _) + { + &usage("$0: repos directory `$repos' is not a directory."); + } + +# Check that all of the regular expressions can be compiled and +# compile them. +{ + my $ok = 1; + for (my $i=0; $i<@project_settings_list; ++$i) + { + my $match_regex = $project_settings_list[$i]->{match_regex}; + + # To help users that automatically write regular expressions + # that match the root directory using ^/, remove the / character + # because subversion paths, while they start at the root level, + # do not begin with a /. + $match_regex =~ s#^\^/#^#; + + my $match_re; + eval { $match_re = qr/$match_regex/ }; + if ($@) + { + warn "$0: -m regex #$i `$match_regex' does not compile:\n$@\n"; + $ok = 0; + next; + } + $project_settings_list[$i]->{match_re} = $match_re; + } + exit 1 unless $ok; +} + +# Harvest common data needed for both commit or revprop-change. + +# Figure out what directories have changed using svnlook. +my @dirschanged = &read_from_process($svnlook, 'dirs-changed', $repos, + '-r', $rev); + +# Lose the trailing slash in the directory names if one exists, except +# in the case of '/'. +my $rootchanged = 0; +for (my $i=0; $i<@dirschanged; ++$i) + { + if ($dirschanged[$i] eq '/') + { + $rootchanged = 1; + } + else + { + $dirschanged[$i] =~ s#^(.+)[/\\]$#$1#; + } + } + +# Figure out what files have changed using svnlook. +my @svnlooklines = &read_from_process($svnlook, 'changed', $repos, '-r', $rev); + +# Parse the changed nodes. +my @adds; +my @dels; +my @mods; +foreach my $line (@svnlooklines) + { + my $path = ''; + my $code = ''; + + # Split the line up into the modification code and path, ignoring + # property modifications. + if ($line =~ /^(.). (.*)$/) + { + $code = $1; + $path = $2; + } + + if ($code eq 'A') + { + push(@adds, $path); + } + elsif ($code eq 'D') + { + push(@dels, $path); + } + else + { + push(@mods, $path); + } + } + +# Declare variables which carry information out of the inner scope of +# the conditional blocks below. +my $subject_base; +my $subject_logbase; +my @body; +# $author - declared above for use as a command line parameter in +# revprop-change mode. In commit mode, gets filled in below. + +if ($mode eq 'commit') + { + ###################################################################### + # Harvest data using svnlook. + + # Get the author, date, and log from svnlook. + my @infolines = &read_from_process($svnlook, 'info', $repos, '-r', $rev); + $author = shift @infolines; + $date = shift @infolines; + shift @infolines; + my @log = map { "$_\n" } @infolines; + + ###################################################################### + # Modified directory name collapsing. + + # Collapse the list of changed directories only if the root directory + # was not modified, because otherwise everything is under root and + # there's no point in collapsing the directories, and only if more + # than one directory was modified. + my $commondir = ''; + my @edited_dirschanged = @dirschanged; + if (!$rootchanged and @edited_dirschanged > 1) + { + my $firstline = shift @edited_dirschanged; + my @commonpieces = split('/', $firstline); + foreach my $line (@edited_dirschanged) + { + my @pieces = split('/', $line); + my $i = 0; + while ($i < @pieces and $i < @commonpieces) + { + if ($pieces[$i] ne $commonpieces[$i]) + { + splice(@commonpieces, $i, @commonpieces - $i); + last; + } + $i++; + } + } + unshift(@edited_dirschanged, $firstline); + + if (@commonpieces) + { + $commondir = join('/', @commonpieces); + my @new_dirschanged; + foreach my $dir (@edited_dirschanged) + { + if ($dir eq $commondir) + { + $dir = '.'; + } + else + { + $dir =~ s#^\Q$commondir/\E##; + } + push(@new_dirschanged, $dir); + } + @edited_dirschanged = @new_dirschanged; + } + } + my $dirlist = join(' ', @edited_dirschanged); + + ###################################################################### + # Assembly of log message. + + if ($commondir ne '') + { + $subject_base = "r$rev - in $commondir: $dirlist"; + } + else + { + $subject_base = "r$rev - $dirlist"; + } + my $summary = @log ? $log[0] : ''; + chomp($summary); + $subject_logbase = "r$rev - $summary"; + + # Put together the body of the log message. + push(@body, "Author: $author\n"); + push(@body, "Date: $date\n"); + push(@body, "New Revision: $rev\n"); + push(@body, "\n"); + if (@adds) + { + @adds = sort @adds; + push(@body, "Added:\n"); + push(@body, map { " $_\n" } @adds); + } + if (@dels) + { + @dels = sort @dels; + push(@body, "Removed:\n"); + push(@body, map { " $_\n" } @dels); + } + if (@mods) + { + @mods = sort @mods; + push(@body, "Modified:\n"); + push(@body, map { " $_\n" } @mods); + } + push(@body, "Log:\n"); + push(@body, @log); + push(@body, "\n"); + } +elsif ($mode eq 'revprop-change') + { + ###################################################################### + # Harvest data. + + my @svnlines; + # Get the diff file if it was provided, otherwise the property value. + if ($diff_file) + { + open(DIFF_FILE, $diff_file) or die "$0: cannot read `$diff_file': $!\n"; + @svnlines = ; + close DIFF_FILE; + } + else + { + @svnlines = &read_from_process($svnlook, 'propget', '--revprop', '-r', + $rev, $repos, $propname); + } + + ###################################################################### + # Assembly of log message. + + $subject_base = "propchange - r$rev $propname"; + + # Put together the body of the log message. + push(@body, "Author: $author\n"); + push(@body, "Revision: $rev\n"); + push(@body, "Property Name: $propname\n"); + push(@body, "\n"); + unless ($diff_file) + { + push(@body, "New Property Value:\n"); + } + push(@body, map { /[\r\n]+$/ ? $_ : "$_\n" } @svnlines); + push(@body, "\n"); + } + +# Cached information - calculated when first needed. +my @difflines; + +# Go through each project and see if there are any matches for this +# project. If so, send the log out. +foreach my $project (@project_settings_list) + { + my $match_re = $project->{match_re}; + my $match = 0; + foreach my $path (@dirschanged, @adds, @dels, @mods) + { + if ($path =~ $match_re) + { + $match = 1; + last; + } + } + + next unless $match; + + my @email_addresses = @{$project->{email_addresses}}; + my $userlist = join(' ', @email_addresses); + my $to = join(', ', @email_addresses); + my $from_address = $project->{from_address}; + my $hostname = $project->{hostname}; + my $log_file = $project->{log_file}; + my $reply_to = $project->{reply_to}; + my $subject_prefix = $project->{subject_prefix}; + my $summary = $project->{summary}; + my $diff_wanted = ($project->{show_diff} and $mode eq 'commit'); + my $stdout = $project->{stdout}; + + my $subject = $summary ? $subject_logbase : $subject_base; + if ($subject_prefix =~ /\w/) + { + $subject = "$subject_prefix $subject"; + } + my $mail_from = $author; + + if ($from_address =~ /\w/) + { + $mail_from = $from_address; + } + elsif ($hostname =~ /\w/) + { + $mail_from = "$mail_from\@$hostname"; + } + elsif (defined $smtp_server and ! $stdout) + { + die "$0: use of either `-h' or `--from' is mandatory when ", + "sending email using direct SMTP.\n"; + } + + my @head; + my $formatted_date; + if ($stdout) + { + $formatted_date = strftime('%a %b %e %X %Y', localtime()); + push(@head, "From $mail_from $formatted_date\n"); + } + $formatted_date = strftime('%a, %e %b %Y %X %z', localtime()); + push(@head, "Date: $formatted_date\n"); + push(@head, "To: $to\n"); + push(@head, "From: $mail_from\n"); + push(@head, "Subject: $subject\n"); + push(@head, "Reply-to: $reply_to\n") if $reply_to; + + ### Below, we set the content-type etc, but see these comments + ### from Greg Stein on why this is not a full solution. + # + # From: Greg Stein + # Subject: Re: svn commit: rev 2599 - trunk/tools/cgi + # To: dev@subversion.tigris.org + # Date: Fri, 19 Jul 2002 23:42:32 -0700 + # + # Well... that isn't strictly true. The contents of the files + # might not be UTF-8, so the "diff" portion will be hosed. + # + # If you want a truly "proper" commit message, then you'd use + # multipart MIME messages, with each file going into its own part, + # and labeled with an appropriate MIME type and charset. Of + # course, we haven't defined a charset property yet, but no biggy. + # + # Going with multipart will surely throw out the notion of "cut + # out the patch from the email and apply." But then again: the + # commit emailer could see that all portions are in the same + # charset and skip the multipart thang. + # + # etc etc + # + # Basically: adding/tweaking the content-type is nice, but don't + # think that is the proper solution. + push(@head, "Content-Type: text/plain; charset=UTF-8\n"); + push(@head, "Content-Transfer-Encoding: 8bit\n"); + + push(@head, "\n"); + + if ($diff_wanted and not @difflines) + { + # Get the diff from svnlook. + my @no_diff_deleted = $no_diff_deleted ? ('--no-diff-deleted') : (); + my @no_diff_added = $no_diff_added ? ('--no-diff-added') : (); + @difflines = &read_from_process($svnlook, 'diff', $repos, + '-r', $rev, @no_diff_deleted, + @no_diff_added); + @difflines = map { /[\r\n]+$/ ? $_ : "$_\n" } @difflines; + } + + if ($stdout) + { + print @head, @body; + print @difflines if $diff_wanted; + } + elsif (defined $sendmail and @email_addresses) + { + # Open a pipe to sendmail. + my $command = "$sendmail -f'$mail_from' $userlist"; + if (open(SENDMAIL, "| $command")) + { + print SENDMAIL @head, @body; + print SENDMAIL @difflines if $diff_wanted; + close SENDMAIL + or warn "$0: error in closing `$command' for writing: $!\n"; + } + else + { + warn "$0: cannot open `| $command' for writing: $!\n"; + } + } + elsif (defined $smtp_server and @email_addresses) + { + my $smtp = Net::SMTP->new($smtp_server) + or die "$0: error opening SMTP session to `$smtp_server': $!\n"; + handle_smtp_error($smtp, $smtp->mail($mail_from)); + handle_smtp_error($smtp, $smtp->recipient(@email_addresses)); + handle_smtp_error($smtp, $smtp->data()); + handle_smtp_error($smtp, $smtp->datasend(@head, @body)); + if ($diff_wanted) + { + handle_smtp_error($smtp, $smtp->datasend(@difflines)); + } + handle_smtp_error($smtp, $smtp->dataend()); + handle_smtp_error($smtp, $smtp->quit()); + } + + # Dump the output to logfile (if its name is not empty). + if ($log_file =~ /\w/) + { + if (open(LOGFILE, ">> $log_file")) + { + print LOGFILE @head, @body; + print LOGFILE @difflines if $diff_wanted; + close LOGFILE + or warn "$0: error in closing `$log_file' for appending: $!\n"; + } + else + { + warn "$0: cannot open `$log_file' for appending: $!\n"; + } + } + } + +exit 0; + +sub handle_smtp_error +{ + my ($smtp, $retval) = @_; + if (not $retval) + { + die "$0: SMTP Error: " . $smtp->message() . "\n"; + } +} + +sub usage +{ + warn "@_\n" if @_; + die "usage (commit mode):\n", + " $0 REPOS REVNUM [[-m regex] [options] [email_addr ...]] ...\n", + "usage: (revprop-change mode):\n", + " $0 --revprop-change REPOS REVNUM USER PROPNAME [-d diff_file] \\\n", + " [[-m regex] [options] [email_addr ...]] ...\n", + "options are:\n", + " -m regex Regular expression to match committed path\n", + " --from email_address Email address for 'From:' (overrides -h)\n", + " -h hostname Hostname to append to author for 'From:'\n", + " -l logfile Append mail contents to this log file\n", + " -r email_address Email address for 'Reply-To:'\n", + " -s subject_prefix Subject line prefix\n", + " --summary Use first line of commit log in subject\n", + " --diff y|n Include diff in message (default: y)\n", + " (applies to commit mode only)\n", + " --stdout Spit the message in mbox format to stdout.\n", + "\n", + "This script supports a single repository with multiple projects,\n", + "where each project receives email only for actions that affect that\n", + "project. A project is identified by using the -m command line\n". + "option with a regular expression argument. If the given revision\n", + "contains modifications to a path that matches the regular\n", + "expression, then the action applies to the project.\n", + "\n", + "Any of the following email addresses and command line options\n", + "(other than -d) are associated with this project, until the next -m,\n", + "which resets the options and the list of email addresses.\n", + "\n", + "To support a single project conveniently, the script initializes\n", + "itself with an implicit -m . rule that matches any modifications\n", + "to the repository. Therefore, to use the script for a single-\n", + "project repository, just use the other command line options and\n", + "a list of email addresses on the command line. If you do not want\n", + "a rule that matches the entire repository, then use -m with a\n", + "regular expression before any other command line options or email\n", + "addresses.\n", + "\n", + "'revprop-change' mode:\n", + "The message will contain a copy of the diff_file if it is provided,\n", + "otherwise a copy of the (assumed to be new) property value.\n", + "\n"; +} + +# Return a new hash data structure for a new empty project that +# matches any modifications to the repository. +sub new_project +{ + return {email_addresses => [], + from_address => '', + hostname => '', + log_file => '', + match_regex => '.', + reply_to => '', + subject_prefix => '', + show_diff => 1, + stdout => 0}; +} + +sub parse_boolean +{ + if ($_[0] eq 'y') { return 1; }; + if ($_[0] eq 'n') { return 0; }; + + die "$0: valid boolean options are 'y' or 'n', not '$_[0]'\n"; +} + +# Start a child process safely without using /bin/sh. +sub safe_read_from_pipe +{ + unless (@_) + { + croak "$0: safe_read_from_pipe passed no arguments.\n"; + } + + my $openfork_available = $^O ne "MSWin32"; + if ($openfork_available) # We can fork on this system. + { + my $pid = open(SAFE_READ, '-|'); + unless (defined $pid) + { + die "$0: cannot fork: $!\n"; + } + unless ($pid) + { + open(STDERR, ">&STDOUT") + or die "$0: cannot dup STDOUT: $!\n"; + exec(@_) + or die "$0: cannot exec `@_': $!\n"; + } + } + else # Running on Windows. No fork. + { + my @commandline = (); + my $arg; + + while ($arg = shift) + { + $arg =~ s/\"/\\\"/g; + if ($arg eq "" or $arg =~ /\s/) { $arg = "\"$arg\""; } + push(@commandline, $arg); + } + + # Now do the pipe. + open(SAFE_READ, "@commandline |") + or die "$0: cannot pipe to command: $!\n"; + } + my @output; + while () + { + s/[\r\n]+$//; + push(@output, $_); + } + close(SAFE_READ); + my $result = $?; + my $exit = $result >> 8; + my $signal = $result & 127; + my $cd = $result & 128 ? "with core dump" : ""; + if ($signal or $cd) + { + warn "$0: pipe from `@_' failed $cd: exit=$exit signal=$signal\n"; + } + if (wantarray) + { + return ($result, @output); + } + else + { + return $result; + } +} + +# Use safe_read_from_pipe to start a child process safely and return +# the output if it succeeded or an error message followed by the output +# if it failed. +sub read_from_process +{ + unless (@_) + { + croak "$0: read_from_process passed no arguments.\n"; + } + my ($status, @output) = &safe_read_from_pipe(@_); + if ($status) + { + return ("$0: `@_' failed with this output:", @output); + } + else + { + return @output; + } +} Index: branches/fc20-dev/locker/sbin/commit-zephyr =================================================================== --- branches/fc20-dev/locker/sbin/commit-zephyr (revision 2523) +++ branches/fc20-dev/locker/sbin/commit-zephyr (revision 2523) @@ -0,0 +1,52 @@ +#!/bin/bash +# +# This is a script that can be called from a Subversion post-commit hook +# to zephyr a summary of the commit or the full commit. +# +# Use by putting something like the following in hooks/post-commit: +# REPOS="$1" +# REV="$2" +# /mit/snippets/svn-hooks/commit-zephyr "$REPOS" "$REV" -c scripts +# /mit/snippets/svn-hooks/commit-zephyr "$REPOS" "$REV" --full -c scripts-auto -i commits + +export LC_ALL=en_US.UTF-8 + +CLASS=test +INSTANCE=@ +FULL=0 + +OPTS=$(getopt -o c:i:f -l class:,instance:,full -n "$0" -- "$@") || exit $? +eval set -- "$OPTS" +while :; do + case "$1" in + -c|--class) CLASS=$2; shift 2;; + -i|--instance) INSTANCE=$2; shift 2;; + -f|--full) FULL=1; shift;; + --) shift; break;; + *) exit 1;; + esac +done +[ $# -ge 2 ] || exit 1 +REPOS=$1 +REV=$2 + +if [ "$INSTANCE" = "${INSTANCE%@}@" ]; then + INSTANCE=${INSTANCE%@}r$REV +fi + +dirs=$(svnlook dirs-changed "$REPOS" -r "$REV") +svnlook info "$REPOS" -r "$REV" | ( + read -r author + read -r datestamp + read -r logsize + log=$(cat) + echo "r$REV by $author $datestamp" + echo "$log" + svnlook changed "$REPOS" -r "$REV" + if [ "$FULL" -eq 1 ]; then + echo + svnlook diff "$REPOS" -r "$REV" + else + echo svnlook diff "$REPOS" -r "$REV" + fi +) | zwrite -d -c "$CLASS" -i "$INSTANCE" -O "auto" -s "SVN: r$REV" Index: branches/fc20-dev/locker/sbin/delete-user =================================================================== --- branches/fc20-dev/locker/sbin/delete-user (revision 2523) +++ branches/fc20-dev/locker/sbin/delete-user (revision 2523) @@ -0,0 +1,89 @@ +#!/usr/bin/python +import ldap +import ldap.filter +import pwd +import sys + +actuallyRun = False + +def delete_record(l, full_name): + if actuallyRun: + print "Deleting %s..." % (full_name,) + l.delete_s(full_name) + else: + print "Would have deleted %s" % (full_name,) + +def findUser(l, username): + # Try to delete the + user_record, = ll.search_s( + "ou=People,dc=scripts,dc=mit,dc=edu", + ldap.SCOPE_SUBTREE, + ldap.filter.filter_format( + "(&(objectClass=posixAccount)" + + "(uid=%s))", + [username])) + + return user_record + +def findGroup(l, username): + group_record, = ll.search_s( + "ou=Groups,dc=scripts,dc=mit,dc=edu", + ldap.SCOPE_SUBTREE, + ldap.filter.filter_format( + "(&(objectClass=posixGroup)" + + "(cn=%s))", + [username])) + + return group_record + +def findApacheConfig(l, uid): + host_records = ll.search_s( + "ou=VirtualHosts,dc=scripts,dc=mit,dc=edu", + ldap.SCOPE_SUBTREE, + ldap.filter.filter_format( + "(&(objectClass=apacheConfig)" + + "(apacheSuexecUid=%s))", + [uid])) + + return host_records + +def findVhost(l, full_name): + host_records = ll.search_s( + "ou=VirtualHosts,dc=scripts,dc=mit,dc=edu", + ldap.SCOPE_SUBTREE, + ldap.filter.filter_format( + "(&(objectClass=scriptsVhost)" + + "(scriptsVhostAccount=%s))", + [full_name])) + + return host_records + +if __name__ == '__main__': + (self, user) = sys.argv + + print "Binding to ldap..." + + ll = ldap.initialize("ldapi://%2fvar%2frun%2fslapd-scripts.socket/") + ll.simple_bind_s("cn=Directory Manager", open('/etc/signup-ldap-pw').read()) + + print "Finding user '%s'..." % (user,) + user_record = findUser(ll, user) + + print "Finding group '%s'..." % (user,) + group_record = findGroup(ll, user) + + print "Searching for apache configurations..." + apache_configs = findApacheConfig(ll, user_record[1]['uidNumber'][0]) + + print "Searching for vhost configurations..." + vhost_configs = findVhost(ll, user_record[0]) + + print "Deleting..." + delete_record(ll, user_record[0]) + delete_record(ll, group_record[0]) + + for config in apache_configs: + delete_record(ll, config[0]) + + for vhost in vhost_configs: + delete_record(ll, vhost[0]) Index: branches/fc20-dev/locker/sbin/get-ldap-admins =================================================================== --- branches/fc20-dev/locker/sbin/get-ldap-admins (revision 2523) +++ branches/fc20-dev/locker/sbin/get-ldap-admins (revision 2523) @@ -0,0 +1,3 @@ +#!/bin/sh +# This script can be run on or off of scripts +ldapsearch -x -h scripts.mit.edu -b dc=scripts,dc=mit,dc=edu -LLL 'cn=Directory Administrators' uniqueMember Index: branches/fc20-dev/locker/sbin/get-versions.pl =================================================================== --- branches/fc20-dev/locker/sbin/get-versions.pl (revision 2523) +++ branches/fc20-dev/locker/sbin/get-versions.pl (revision 2523) @@ -0,0 +1,21 @@ +#!/usr/bin/perl + +system("/mit/scripts/sec-tools/get-passwd.sh"); +system("/mit/scripts/sec-tools/parallel-find.pl"); +sleep 5; + +while(1) { + my $count = `ps -ef | grep find | grep $ENV{USER} | grep -v ps | grep -v grep | wc -l | tr -d '\n'`; + if ($count eq '0') { + last; + } + else { + print "Current have $count find processes running. Please wait.\n"; + sleep 1; + } +} + +print "Done finding files\n"; +system("cat /mit/scripts/sec-tools/store/versions/* >| /mit/scripts/sec-tools/store/scripts-versions"); +print "Done\n"; +#print `cat /mit/scripts/sec-tools/store/versions/`; Index: branches/fc20-dev/locker/sbin/mail-owners.pl =================================================================== --- branches/fc20-dev/locker/sbin/mail-owners.pl (revision 2523) +++ branches/fc20-dev/locker/sbin/mail-owners.pl (revision 2523) @@ -0,0 +1,53 @@ +#!/usr/athena/bin/perl + +use strict; + +use warnings; + +open LIST, "actual"; + +open TEMPLATE, "wordpress-email"; + +my $template = do {local $/;