This contains information for a species delimitation course offered at U. of Gothenburg, Sweden, in Dec. 2010. I am presenting on the methods contained in Brownie.
Local time Gothenburg, Sweden:
I will be delivering the lecture (PDF, Apple Keynote) and running the exercise using EVO. Students may log in to ask questions, show problems they are having, etc. I have also set up a mibbit irc channel (below) [now closed] for discussions during the lecture or exercise (advantage over EVO is that it doesn't require registration, but it also means I can't see your screen).
Exercise:
Objectives:
- Learn how to get help in Brownie.
- Learn how to run Brownie for species delimitation using defaults.
- Learn how to interpret and present results.
- Learn how to change settings and their effect.
- Learn about NEXUS file structure.
- Learn how to make batch files in NEXUS.
- Experiment
What to do:
Note that commands and program output are in fixed-width font.
Let me know if you have questions, fall behind, etc. THIS IS NORMAL. Remember, this course is for YOU -- if something is not going well, it is our duty to fix it.
- Log in to the cluster
- cp -r /usr/local/files/browniefiles ~ in order to copy (cp) the folder (-r to say to copy a folder, /usr/local/files/browniefiles for the actual folder) to our home directory (~). This way, everyone is not trying to save things to the same folder. Note that on an American keyboard, ~ is to the left of the "1" button, but I don't know about other keyboard layouts.
- Start brownie: /usr/local/bin/brownie.sh (there will be a warning based on lack of an input argument: this is fine, it is due to the particular shell script we're using)
- Brownie is being actively developed. You can see the version number (V2.1.3) as well as what the revision number this is in Brownie's source code repository (r302). The copy we are using was compiled on Nov. 12. Note that like much open source software, Brownie uses existing libraries, such as the GNU Scientific Library and code from Rod Page and Paul Lewis. It's also open: you can look at the raw source code, modify it, and make changes to it. The changes can be added to Brownie itself or you can put the code into a new program. Much new scientific software, including some of the others in this course, are written in this way.
- What to do next? When in doubt, get help. Type help, then return.
- Brownie has commands for lots of different functions. Which ones will we want to use?
- Ready? We'll want to use log, echo, hs, and quit. Log stores the output Brownie returns to the screen. Echo stores the input you give Brownie: this can be handy if you're playing around with an analysis and want to remember what you did. Hs does an heuristic search for the best delimited species tree (I played with the possibility of doing an exhaustive search but didn't implement it -- something you could add to the code (hope you have a fast computer to run it on, though). Quit leaves the program. Brownie reports citations for approaches it thinks you have used during an analysis (but these might not be exhaustive).
- Let's find out how to use log. Most Brownie commands give help if you type the command name followed by a space and a question mark. Type log ?.
- Start logging to a file called log1.txt
- Now get help with hs. How to do this?
- Okay, let's quit brownie
- Now, navigate to your directory (folder) that has the brownie input files. cd ("Choose Directory") is how to navigate to a folder. In our case, cd ~/browniefiles
- Make sure you're in the right directory: type ls to see the files here. You should see test1.nex. That's the file we're going to run.
- We're going to start brownie with an input file. As we learned yesterday, jobs are being run on a cluster -- a network of many computers. Type cat /usr/local/bin/brownie.sh to see the file used to start the brownie job. You can see it's very simple, just calling another script. cat /usr/local/sge/browniei.sh will show you this other script: also pretty simple, just with some options for dealing with the batching software. On your own computer, if brownie were installed correctly, just brownie filename.nex would work fine.
- Let's go! brownie.sh test1.nex
- That was underwhelming. All it did was tell us we have 10 trees. What should we do?
- Yes, let's start a log file. log file=test1.log. One thing to note is that we could have named it anything we wanted: test2.pdf, structurama.input, etc. It's still going to be a text file.
- Yes, let's start an echo file, too. echo file=test1.echo
- Make sure we know which options to use: hs ?
- Make it so: hs
- What is going on?
- Once it is done running, quit the program
- Where is the output? ls
- Normally, it would be here, but it is actually one level up: cd ..
- Let's look at the echo file first: less test1.echo
- q to quit less. Now less test1.log
- The tree returned is the species tree. You can open it in something like figtree
- Leave this file and look at other files in the directory. What is besttrees.tre.mesquite.nex? It's a file that, if you opened it in Mesquite, you could look at all the gene trees within the species tree. Rather than have you try to do scp and get Mesquite running, I will demonstrate it now. See this image for where to show constrained trees and here for output.
- Okay, let's look at other options. A key parameter is the weight between the structure cost and the GTP cost. Set this to a high value to put more emphasis on structure (will this tend to split or lump taxa?). Put this low to turn Brownie into a program for doing tree inference under GTP. Play with this.
- So far, we've just done the nonparametric method. What if we wanted to do the parametric method? This is based on ideas from Knowles and Carstens, and just tries to find the assignment of samples to species and the species tree with the best AIC score (where the likelihood is based on the probability of the gene trees given the species tree). Calculating the likelihood is possible (STEM does this, for one -- you'll be learning about this later this week). Brownie can use one of two other programs for this: COAL (Degnan and Salter, 2005) or ms (Hudson 2002). COAL works as you'd expect: give it a species tree and a gene tree, it returns a likelihood. When I was developing Brownie, there was one particular quirk of COAL (a very rare issue, but one sometimes you'd hit by looking over many tree combinations) that made it somewhat problematic in this use, so I opted to use ms as the primary way of calculating gene tree likelihoods. The way this was done was fairly kludgy: the probability of a gene tree is the expected amount of times you'd see it under a given species tree, so Brownie just uses ms to simulate a lot of gene trees under a potential species tree to estimate their probability in this way, rather than calculating the likelihood explicitly using equations from Degnan and Salter (2005). Later this week, you'll be hearing from people with many more elegant approaches to this, but you can try Brownie's approach now. Let's do another heuristic search, but with ms=yes, a different file name for the best trees, and nreps dropped down to 1 for speed. hs ? for help, remember.
- Now let's look at the input files for brownie. Make a copy of the test1.nex file (cp test1.nex test1modified.nex) and open it in your new favorite editor, nano (nano test1modified.nex).
- Walk through NEXUS file structure
- Let's add a Brownie block...
begin brownie;
log file=testbatch.log;
hs;
quit;
end;
- Run the modified file
- Now play with some of the other files in ~/browniefiles and some of the other settings in Brownie. If you're logged into EVO, you can share your screen so I can look over your shoulder to see what you are doing.
- Questions?
