You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
CS141 ADMINISTRATION MANUAL - Grissess
This manual is a point of reference for managing CS141 on Odin.
- INPUT FORMATS
- STUDENT CSV COMMANDS
- USERNAME COMMANDS
- THE LAB SUPER-COMMAND
- ADDING USERS
- PREPARING ASSIGNMENTS
- MANAGING ASSIGNMENTS
Almost all of these scripts accept their input via stdin, the standard input
file descriptor; from most shells, this can be arranged by redirection. For
example, if running the script "bar.sh" with arguments "a" and "b" and with
input "/path/to/foo.csv", while working in this directory:
$ ./bar.sh a b < /path/to/foo.csv
...where "$" indicates the shell's prompt (which may be "#" for root).
The format of a "student CSV", to be mentioned below, is:
where "username" is valid on Odin, even if different from email (which may be
absent). Email should be full--id est, it should end with @domain.com.
There are two general classes of command: those that take a student CSV, and
those that don't; they are recorded in the sections below.
STUDENT CSV COMMANDS
These commands accept, usually on stdin, a CSV file in the "student CSV" format
described in `INPUT FORMATS`. In general, these commands are used for account
management, or for scraping this data for use in the scripts which accept a
list of usernames--most don't otherwise need this information. For this reason,
there are few.
- usernames.sh: Given a CSV as either $1 or on stdin, returns one username per
- make_users.sh: Given a CSV on stdin, creates new users with the given
usernames. Passwords are set to a default in this script, then expired.
Permissions on home directories are "fixed", but no "lab" links are made; see
The following files all *take a list of usernames on stdin*. This is most
conveniently generated by running usernames.sh on a students CSV, and piping it
- fix_perms.sh: Ensure that student home directories have the correct permissions.
- expired.sh: Detect users whose passwords have been expired with passwd -e; id
est, newly created accounts who haven't set the password from the default.
- nonexistent.sh: Detect users that don't exist on Odin. Useful for detecting
adds to a student CSV, e.g. during add/drops.
- lsrf.sh: Reports the number of (real) files rooted in a user's home
directory. Used for the intro-to-Odin assignment. Zeroes are colored for
- udu.sh: Reports (apparent) disk usage in the home directories of users given
on stdin. The values are given using du with -h and -b; M and G units are
- ucp.sh: Copies a file to all users' home directories. The first argument is a
file to copy; the second is an optional path relative to the home directory
to copy to; if it does not end with a slash, it will specify the basename of
the destination (the default is "./"--id est, directly into the root of their
home directory, and with the same base name as the source). If the third
argument is "force", the file is copied over whatever may already be there.
After a successful copy, full ownership is transferred to the user.
- usu.sh: Runs its arguments as each user. Note that arguments are passed to su
as "$*", which tends to interfere with shell quoting--avoid this if possible.
Anything beyond trivial commands should be made into a script which is
invoked by this command--ensure that such a script and all containing
directories are r-x by the users! Note also that commands are invoked with
user privilege, and so this cannot be used to perform administrative changes.
- lab.sh: The "labs" super-command. See `THE LAB SUPER-COMMAND` below.
THE LAB SUPER-COMMAND
Running "lab.sh" gives access to a wide variety of commands for managing
assignments and labs for courses. These commands expect a "COURSE" environment
variable to distinguish the directories in which they operate; this will be
referred to in the documentation as "$COURSE" where its value is used. In
general, ./lab.sh FOO BAR BAZ is the same as invoking ./lab_FOO.sh BAR BAZ with
the same stdin. As with the `USERNAME COMMANDS` above, all lab_* commands
expect username to operate on in stdin. The following are defined:
- lab.sh mkstudents: Make the student submission directories in /course/$COURSE/assn/.
- lab.sh fixlinks: Attempts to make a link to each student's submission
directory in their home directory. This will fail if the student already has
a "/course/$COURSE" that isn't already a link. The next command will help in diagnosing
- lab.sh badlinks: Emits usernames with no "$COURSE" link in their home
directory. This includes nonexistence (before a fixlinks, e.g.) or if the
user has another object named "$COURSE".
- lab.sh deadlinks: Emits usernames with an extant "$COURSE" link that points to
something that doesn't exist.
- lab.sh test: Tests the lab.sh command.
- lab.sh mkassn ASSN [ASSN...]: Makes the assignment ASSN. This creates a
directory in all users' submission directories and sets the permissions
properly. The assignment is initially *closed*.
- lab.sh assnopen ASSN [ASSN...]: Opens the assignments for the given users to
- lab.sh assnclose ASSN [ASSN...]: Closes the assignments for the given users.
- lab.sh rmassn ASSN [ASSN...]: Removes assignment directories. This succeeds
only if the assignment directories are empty.
- lab.sh assnstat ASSN [ASSN...]: Determines the status of the assignments for
the given users. This is usually open or closed, but this command will deeply
inspect the ACLs to determine if there are issues, as well as impersonate
users and use the "test -w" builtin to determine the system's writability,
and warn you about problems. It should be a first resort for any complaints
that submission "wasn't working" insofar as it explicitly tests all relevant
- lab.sh lsassn : Lists all assignments amongst the given users, and prints
totals at the end. If the literal "1" is given as its only argument, it
produces a "short" output which contains only the newline-separated union of
all encountered assignments, which is suitable for, e.g., command expansion
to pass as a list of assignments to other lab commands.
- lab.sh assnlsrf [-l LIM] ASSN [ASSN...]: Lists the number of real files in an
assignment per user, much like lsrf.sh. Zeroes are color-coded in red, and
assignment counts below the optionally-given limit are colored yellow for
quick recognition. This makes for a good way to investigate current progress
with an assignment.
- lab.sh assndu ASSN [ASSN...]: Lists the sizes of the trees in each user's
assignment folder. M and G are highlighted, as with udu.sh.
- lab.sh assnchown ASSN [ASSN...]: Changes the ownership of all of the files
in the assignment directories to the user, and ensures that the permissions
on the directory is correct. This *may* close the directory, after which you
may want to assnopen it. This is generally used to correct files owned by
root:root in the directories.
- lab.sh fixperms: Changes ownership and permission on lab directories such
that users have rx to their own directories. This may be necessary after
catastrophic permission issues (as have happened before), but should
otherwise be unnecessary.
This part of the manual will give quick advice on how to start with these
Presume that you have received a file "students.csv" somewhere in the standards
students CSV format. To start, we will create the new user accounts:
$ ./make_users.sh < students.csv
Then, after that is done, fix the permissions on the home directories:
$ ./usernames.sh students.csv | ./fix_perms.sh
If, for some reason, you need to add students later, just edit students.csv and
insert the lines. This will tell you which users where added (or, rather, which
don't already exist):
$ ./usernames.sh students.csv | ./nonexistent.sh
If that is non-empty for some reason, go ahead and do the two lines at the
beginning of this section again. make_users.sh is smart enough to not repeat
work, and fix_perms.sh, while redundant, shouldn't negatively affect anything.
First, ensure that a course directory exists for your purposes; I'm using the
fake course name "cs404-s17", but you should replace a name of your own:
$ export COURSE=cs404-s17
$ mkdir -p /course/$COURSE/assn
Afterward, Make sure all users have a submission directory:
$ ./usernames.sh students.csv | ./lab.sh mkstudents
Then make sure that they all have a link to the submissions folder:
$ ./usernames.sh students.csv | ./lab.sh fixlinks
(If, for some reason, that has some errors in it, ./lab.sh badlinks is your
For sanity, you may want to also verify that the links aren't dead:
$ ./usernames.sh students.csv | ./lab.sh deadlinks
If any names are output, you may want to try the first command in this section
As with the section above, if you find you need to add users again, these
commands are all idempotent, and work best after fix_perms.sh.
To make an assignment:
$ ./usernames.sh students.csv | ./lab.sh mkassn AssnName
This creates a "AssnName" directory under each students' submission directory,
so make sure to choose a name that suits this directory. Initially, new
assignments are closed, which you can verify using:
$ ./usernames.sh students.csv | ./lab.sh assnstat AssnName
...you should observe that users can't write to them yet, and that they are
"closed". If you see anything more, that's indicative of a permission error you
should look into.
To open the assignment for the students:
$ ./usernames.sh students.csv | ./lab.sh assnopen AssnName
You can use assnstat as above to verify that users can write to the assignment,
and that it is open. Note also that you don't need to open the assignment for
all users; if you'd like to open it only for a select few users, you can pass
those usernames in to stdin, as usual (e.g., via a file, or by typing them in
line by line). Opening an assignment is, as usual, idempotent.
To close the assignment for everyone:
$ ./usernames.sh students.csv | ./lab.sh assnclose AssnName
As above, you can use assnstat to verify this, and you need not do this for all
It's fairly easy to check progress with an assignment; for example, assume we
want to check how the class is doing with an assignment that is supposed to
include three files:
$ ./usernames.sh students.csv | ./lab.sh assnlsrf AssnName 3
...this will highlight assignment directories that contain 0 files, as well as
those containing less than 3 (in different colors). Note that it's not specific
as to *which* three files are there--for that, you need to actually grade them
:) . Nonetheless, this is a useful litmus test for current progress.
To see what assignments are presently available to students:
$ ./usernames.sh students.csv | ./lab.sh lsassn
And, finally, if you need to remove an assignment:
$ ./usernames.sh students.csv | ./lab.sh rmassn AssnName
Note that this will not remove non-empty directories.
This section documents information of use to administrators and maintainers of
The following directories are important to the system:
- /home: contains the home directories of the administrators and other
privileged users of the system.
- /home/students: contains the home directories of the students and other
clients of the system.
- /course/$COURSE: contains data and work specific to the $COURSE course.
- /course/$COURSE/assn: contains assignments managed by lab.sh; see `THE LAB
SUPER-COMMAND`, above. A link to this directory is usually placed by "lab.sh
fixlinks" in each student's home directory.
These scripts are meant to run as root, so as to bypass permission checks. In
general, it must be run by a user with capabilities to:
- CAP_CHOWN: Change file and directory owning groups and users arbitrarily.
- CAP_DAC_OVERRIDE: Read and write files arbitrarily, regardless of
destination directory permission (and bypassing all EACCESS errors
encountered in path walking; see "Step 2" in path_resolution(7)).
- CAP_FOWNER: Change properties of files normally reserved to the owner (such
as mode bits--chown(2)--and ACLS--acl(5))
- ...and possibly others; this list isn't tested.
We will define the following terms:
- root:admin, the "administrator's credential", is used to permit sysadmins
permission to an object without becoming root;
- root:root, the "root credential", is the owner of the filesystem root and
many other critical directories;
- u+rwx, g+rx, o+rx, is the "default directory permission", which is the one
usually used by mkdir as a user;
- u+rw, g+r, o+r, is the "default file permission", which is usually assigned
(by umask) to whatever files are created;
- o-rwx is a "restricted" permission, intended to avoid world-readability.
Permissions for the following directories are maintained as follows.
- /home: root credential, default directory permission.
- /home/students: admin credential, default directory permission.
- /home/students/$USER: $USER:admin, restricted default directory permission
(to allow admins to read homes without becoming root, but not permit other
users to read such directories).
- /course/$COURSE: admin credential, default directory permission.
- /course/$COURSE/assn: idem
- /course/$COURSE/assn/$USER: $USER:admin, restricted default directory
- /course/$COURSE/assn/$USER/$ASSN: admin credential, restricted default
directory permission, but with acls:
- u:$USER:rx, if the directory is closed, or u:$USER:rwx if it is open;
- d:u::rwx and d:g::rwx--no masked permission bits for entries created inside
- d:g:admin:rwx--group admin can edit any children;
- d:o:0--no default permissions for created entries.
...entries under /course/$COURSE/assn/$USER/ are thereby assigned, by
default, the given "d:" (default) ACL as above, restricted by the umask
(usually the "default" permissions above). See acl(5) for more details.
For modifiers and maintainers of these programs, I have one recommendation: be
as conservative as possible with any modifications. All of these utilities
check almost every precondition before actually performing an action (e.g., not
creating through links, not opening a directory already open, etc.), and this
has saved me on occasion. It is far easier to deal with some harmless bugs due
to overcaution, rather than have to recover permissions of hundreds of
directories while students are working.
As a corollary, do not provide defaults if none are certainly correct, and
definitely not if a default might lead to unsafe behavior. $COURSE handling,
for example, intentionally is not defaulted.
Always have "stat" tools, and be extra pedantic with their checking--see, for
example, "lab_assnstat.sh", which checks for various conditions that "shouldn't
happen" but nonetheless proceed to in practice. These are your first line of
defense against misconfigurations.