Do you know a bit of Perl? Ensembl hosts an API (Application Programmers Interface) which uses Object-Oriented Perl to extract data from Ensembl databases. This API is public and can be used for people to programmatically access the data in the Ensembl database. We understand that not everyone is used to Object-Oriented code, although people may have basic Perl skills and be interested in using our datasets. For that kind of bioinformaticist, I would recommend a recent short read in O’Reilly’s Broadcast:

Beginners Introduction to Object-Oriented Programming with Perl – O’Reilly Broadcast

And for the more advanced readers, the classic reference book in OO-Perl would be Damian Conway’s Object Oriented Perl, which a part from being very informative, has a really cool cover 🙂

We are always trying to lower the barrier to entry for research communities interested in using the Ensembl database in programmatic ways that make use of all the complexity associated with the generation of our data. That’s why our API is public and well-documented. You can learn about our API by attending on of our API workshops for free (e.g.: 1-3 December – Univ. Cambridge, UK). We are currently trying to smooth things out even more, working on ways to make it even easier to download all that’s needed to use the API and have the example scripts running in your computer with the minimum number of steps. Keep tuned for news in this respect soon…

Ensembl has begun to incorporate data from genome-wide association studies. These data are being added in coordination with the European Genotype Archive, a new database resource at the EBI designed to provide a permanent archive for human variation data that is not available for unlimited public release because of ethical or individual privacy restrictions. The European Genotype Archive has recently launched with the raw data from the Wellcome Trust Case Control Consortium (WTCCC. 2007. Genome-wide association study of 14,000 cases of seven common diseases and 3,000 shared controls. Nature 447:661-678). In the future the EGA will provide additional array-based genotype data as well as data from re-sequencing and CNV studies. The EGA will also contain phenotype data.

Ensembl is incorporating summary data from genome-wide association studies represented in the EGA. The data generally represent the p-value for each of the tested SNP (Single Nucleotide Polymorphism) associated with the given phenotype.

The WTCCC summary data is now available on Ensembl as DAS tracks selectable from the “DAS Sources” menu from the CytoView and ContigView pages. The following menu items provide access to data from biopolar disorder (BD), coronary artery disease (CAD), cardiovascular disease (CD), hypertension (HT), type 1 diabetes (T1D), type 2 diabetes (T2D):


In future releases, GWAS data will be integrated into the Ensembl variation databases.

We will be adding additional data to both Ensembl and the European Genotype Archive as the data become available. We hope you find these new data resources useful.

I’m in the airline lounge about to head back from “Biology of Genomes” at Cold Spring Harbor Laboratory. As always, it was a great meeting; highlights for me was seeing the 1,000 genomes data starting to flow – it is clear that the shift in technology is going to change the way we think about population genomics – and for me, the best session was one on “non-traditional models” – Dogs, Horses and Cows, where the ability to do cost effective genotyping has completely revolutionised this field. Now the peculiarities of the breeding structures, with Dog breeds being selected for diverse phenotypes, Cows with the elite bulls siring thousands of offspring due to artificial insemination and Horses having obsessive trait fixation over the last 1,000 years can really bring power to genetics in different ways. Expect alot more knowledge to come from these organisms and others (chickens, pigs, sheep…) over the coming years.

For my own group, Daniel Zerbino talked about Velvet, our new short read assembler which has also just been published in Genome Research (link). Velvet is now robust and capable of assembling “lower” eukaryotic genomes – certainly up to 300MB from short reads in read pair format. It is also being extensively used by other groups, often for partial, minature de novo assemblies in regions. It went down well, and Daniel handled some pretty tricky questions in the Q&A afterwards. Next up – we get access to a 1.5TB real memory machine, and put a whole human genome WGS into memory. Alison (Meynert) and Michael (Hoffman) had great posters on cis-regulation and looked completely exhausted at the end of their poster session.

From Ensembl, Javier talked about Enredo-Pecan-Ortheus (which we often nickname as EPO) pipeline. As some said afterwards to us “you’ve really solved the problem, haven’t you” – Javier was able to show clear evidence that each component was working well, better than competitive methods, and having a impact on real biological problems, for example, derived allele frequency. Its ability to handle duplications is a key innovation. Javier and Kathryn are current wrestling in the “final” 2x genomes into this framework, from which point we will start to have a truly comprehensive grasp on mammalian DNA alignments. I also like it as Enredo is another “de bruijn graph” like mechanism. Currently the joke is that about 10 minutes into any conversation I say “well, the right way to solve this problem is to put the DNA sequence into a de bruijn graph”.

Going to CSHL biology of genomes is always a little wince making though as this field – high end genomics – really prefers to use the UCSC Genome Browser (which as I’ve written before on, is a good browser, and I take the use of it to be our challenge to make better interfaces for these users on our side). My informal counting of screen shots was > 20 UCSC, 4 Ensembl (sneaking one case of ‘Ensembl genes’ shown in the UCSC browser as a point for each side) and 0 NCBI shots. Well. It just shows the task ahead of us. e50! – our user interface relaunch – is coming together, and we will start focus-group testing soon – time for us to address our failings head on. I’ll be blogging more about this as we start to head towards broader testing.

Lots more to write about potentially – Neanderthals, Francis Collins singing in the NHGRI band (quite an experience), reduced representation libraries with Elliott, genome wide association studies (of which, I just _love_ the basic phenotype measures, from groups like Manolis Dermitzakis) and structural variation… but for the moment I’ve got to persuade my body to feel as if it is 11.30 at night and see if I can get a good nights sleep on the plane.

Things are moving within the rat community as this month’s Nature Genetics issue shows with a special on rat genetics exploring the latest developments.


  • ENU-induced gene targeting in rats;
  • A ‘white paper’ discussing progress and prospects in rat genetics;
  • A brief overview on rat genome resources online;
  • ENU-induced gene targeting in rats;
  • A contribution on dynamics of CNV in rat and their impact in phenotypes;
  • A survey of genetic variation from The STAR Consortium (over 3 million newly identified SNPs and over 20,000 SNPs genotyped across 167 distinct inbred rat strains);
  • and several papers focusing on the identification of genetic variants associated to rat models of human disease…

The driving force behind these outstanding achievements can be found on a well interliked rat community bridging resources across the Atlantic: RGD and the EURATools Consortium (FP6 contract number LSHG-CT-2005-019015) collaborations are a good example.

EURATools investigators are developing integrated genome tools (Ensembl is one of the partners of this consortium). Integrating high-throughput sequencing and genotyping with informatics; intensive analysis of phenotypes, gene sequence and gene expression in congenic strains to identify genes and regulatory pathways for a wide range of rat disease phenotypes; and establishing optimised protocols for rat gene targeting are the goals of this ambitious EU funded project.

We have four groups on campus interested in human genes: Ensembl, Havana, whos data forms the bulk of the Vega database, HGNC, the human gene nomenclature committee, and finally UniProt, which has a special initiative on human proteins. With all these groups on the hinxton campus, and with all of them reporting to (at least one) of myself (Ewan Birney), Rolf Apweiler or Tim Hubbard, who form the three-way coordination body now called “Hinxton Sequence Forum”, HSF it should all work out well, right?

Which is sort of true; the main thing that has recently changed over the last year has been far, far closer coordination between these four groups than there was ever before, meaning we will be achieving an even closer coordination of our data, leaving us hopefully with the only differences being update cycle and genes which cannot be coordinated fully (eg, due to gaps in the assembly).

Each of these groups have a unique view point on the problem. Ensembl wants to create as best-as-possible geneset across the entire human genome, and its genesis back in 2000 was that this had to be largely automatic to be achievable in the time scale desired, being months (not years) after data was present. Havana wants to provide the best possible individual gene calls when they annotate a region, integrating both computational, high throughput and individual literature references together, UniProt wants to provide maximal functional information on the protein products of genes, using many literature references on protein function which are not directly informative on gene structure and finally HGNC wants to provide a single, unique, symbol for each gene to provide a framework for discussing genes, in particular between practicing scientists.

Three years ago, each group knew of the other’s existence, often discussed things, was friendly enough but rarely tried to understand in depth why certain data items were causing conflicts as they moved between the different groups. Result: many coordinated genes but a rather persistent set of things which was not coordinated. Result of that: irritated users.

This year, this has already changed, and will change even more over 2008 and 2009. Ensembl is now using full length Havana genes in the gene build, such that when Havana has integrated the usually complex web of high throughput cDNAs, ESTs and literature information, these gene structures “lock down” this part of the genome. About one third of the genome has Havana annotation, and because of the ENCODE Scale up award to a consortium headed by Tim Hubbard, this will now both extend across the entire genome and be challenged and refined by some of the leading computational gene finders world wide (Michael Brent, Mark Diekhans and Manolis Kellis, please take a bow). Previously Ensembl brought in Havana on a one-off basis; now this process has been robustly engineered, and Steve Searle, the co-head of the Gene Build team, is confident this can work in a 4-monthly cycle. This means it seems possible that we can promise a worse-case response to a bad gene structure being fixed in six months, with the fixed gene structure also being present far faster on the Vega web site. It also means that the Ensembl “automated” system will be progressively replaced by this expert lead “manual” annotation over the next 3 years across the entire genome.

(An aside. I hate using the words “automated” and “manual” for these two processes. The Ensembl gene build is, in parts, very un-automated, with each gene build being precisely tailored to the genome of interest in a manual manner, by the so called “gene builder”. In contrast “manual” annotation is an expert curator looking at the results of many computational tools, each usually using different experimental information mapped, in often sophisticated ways, onto the genome. Both use alot of human expertise and alot of computational expertise. The “Ensembl” approach is to use human expertise in the crafting of rules, parameters and choosing which evidence is the most reliable in the context of the genome of interest, but having the final decision executed on those rules systematically, whereas the “Havana” curation approach is to use human expertise inherently gene-by-gene to provide the decision making in each case, and have the computational expertise focus on making this decision making as efficient as possible. Both will continue as critical parts of what we do, with higher investment genomes (or gene regions in some genomes) deserving the more human-resource hungry per genome annotated “manual” curation whereas “automated” systems, which still have a considerable human resource, can be scaled across many more genomes easily).

This joint Havana/Ensembl build will, by construction, be both more correct and more stable over time due to the nature of the Havana annotation process. This means other groups interacting with Havana/Ensembl can work in a smoother, more predictable way. In particular on campus it provides a route for the UniProt team to both schedule their own curation in a smart way (basically, being post-Havana curation) and provide a feedback route for issues noticed in UniProt curation which can be fixed in a gene-by-gene manner. This coordination also helps drive down the issues with HGNC. HGNC always had a tight relationship with Havana, providing HGNC names to their structures, but the HGNC naming process did not coordinate so well with the Ensembl models, with gene names in complex cases becoming confused. This now can be untangled at the right levels – when it is an issue with gene structures, prioritise those for the manual route, when it is an issue with the transfer of the assignment of HGNC names (which primarily has individual sequences, with notes to provide disambiguation) to the final Havana/Ensembl gene models this can be triaged and fixed. HGNC will be providing new classifiers of gene names to deal with complex scenarios where there is just no consistent rule-based way of classifying the difference between “gene” “locus” and “transcript” in a way which can work genome-wide. The most extreme example are the ig loci, with a specialised naming scheme for the components of each locus, but there are other oddities in the genome, such as the proto-cadherin locus which is… just complex. By having these flags, we can warn users that they are looking at a complex scenario, and provide the ability for people who want to work only with cases that follow the “simple” rules (one gene, in one location, with multiple transcripts) the ability to work just in that genome space, without pretending that these parts of biology don’t exist.

It also means our relationships to the other groups in this area; in particular NCBI and UCSC (via the CCDS collaboration), NCBI EntrezGenes (via the HGNC collaboration) and other places worldwide can (a) work better with us because we’ve got more of our shop in order and (b) we can provide a system where if we want to change information or a system, we have only one place we need to change it.

End result; far more synchrony of data, far less confusion for users, far better use of our own resources and better integration with other groups. Everyone’s a winner. Although this is all fiddly, sometimes annoying, detail orientated work, it really makes me happy to see us on a path where we can see this resolved.

Last week I was a co-organiser of a Newton Institute workshop on high dimensional statistics in biology. It was a great meeting and there were lots of interesting discussions, in particular on chip-seq methods and protein-DNA binding array work. I also finally heard Peter Bickel talk about the “Genome Structure Correction” method (GSC), something which he developed for ENCODE statistics, which I now, finally, understand. It is a really important advance in the way we think about statistics on the genome.

The headache for genome analysis is that we know for sure that it is a heterogeneous place – lots of things vary, from gene density to GC content to … nearly anything you name. This means that naive parametric statistical measures, for example, assuming everything is poisson, is will completely overestimate the significance. In contrast, naive randomisation experiments, to build some potential empirical distribution of the genome can easily lead to over-dispersed null distributions, ie, end up under estimating the significance (given a choice it is always better to underestimate). What’s nice is that Peter has come up with a sampling method to give you the “right” empirical null distribution. This involves a segmented-block-bootstrap method where in effect you create “feasible” miniature genome samples by sampling the existing data. As well as being intuitively correct, Peter can show it is actually correct given only two assumptions; one that genome’s heterogeneity is block-y at a suitably larger scale than the items being measured, and secondly that the genome has independence of structure once one samples from far enough way, a sort of mixing property. Finally Peter appeals the same ergodic theory used in physics to convert a sampling over space to being a sampling over time; in other words, that by sampling the single genome’s heterogeneity from the genome we have, this produces a set of samples of “potential genomes” that evolution could have created. All these are justifiable, and certainly this is far fewer assumptions than other statistics. Using this method, empirical distributions (which in some cases can be safely assumed to be gaussian, so then far fewer points are needed to get the estimate) can be generated, and test statistics built off these distributions. (Peter prefers confidence limits of a null distribution).

End result – one can control, correctly, for heterogeneity (of certain sorts, but many of the class you want to, eg, gene density). Peter is part of the ENCODE DAC group I am putting together, and Peter and his postdoc, Ben Brown, are going to be making Perl, pseudo-code and R routines for this statistic. We in Ensembl will implement this I think in a web page, so that everyone can easily use this. Overall… it is a great step forward in handling genome-wide statistics.

It is also about as mathematical as I get.

(you wouldn’t expect orangutans out of the trees, would you?)

Ensembl 49 will contain good news on the comparative genomics side. Apart from the new whole-genome multiple alignments for which we can now handle segmental duplications and infer ancestral sequences (see Ewan’s post on the 20th of January), two new species will be available, namely horse and orangutan.

We are especially excited about the new orangutan genome as it is a key species in the primate lineage, in between the human, chimp and gorilla group and the Old World monkeys. Its inclusion in our gene trees will result in a better resolution of the phylogeny of the primate genes.

We’re pleased to announce that Ensembl now has a mirror at the Beijing Genomics Institute, Shenzhen (BGI-SZ). The mirror can be found at

Most of the functionality of the main Ensembl site is mirrored, however we’re still working with our colleagues at the BGI to provide the rest, for example BioMart.

Due to a combination of the volume of data comprising a single Ensembl release (the MySQL data and index files for release 48 take up apround 600Gb, and that’s without counting all f the flat-file dumps) and the very slow Internet connection between the UK and China, the we’re using a “sneakernet” solution – i.e. dumping the data onto a hard drive and shipping it to China. This has proved to be an interesting challenge but it’s working out pretty well so far.

We hope that this mirror will make life easier for our users in and around China. We’re actively trying to set up mirrors elsewhere around the world to reduce network delays and improve peoples’ Ensembl experience; we’ll post here as soon as any new mirrors come online.

I would like to thank our colleagues at the BGI-SZ, particularly Lin Fang, for setting this mirror up.

I’ve just been visiting CNIO in madrid – a great, fancy new(ish) institute in Madrid focusing on cancer – it was a great visit if you ignore the 2 hour delay (thanks Iberia) coming out and currently 1 hour delay (thanks BA…) coming back. They are doing all the things one expects from a high-end molecular biology institute. There are a chip-chip guys, moving to chip-seq. There are some classic cell biologists moving into more genome wide assays (in this case, replication). They have a great prospective sample collection in two cancers, and are about to get into a Genome Wide Association Study (GWAS).

David – the head of bioinformatics service – already is leveraging Ensembl alot. They script against our databases (Perl API mainly) and have a local mirror set up. They ran courses, bringing over Ensembl people for both an API course and a Browser course (contact helpdesk if you’d like this to happen at your institute…). But even then, discussions with David made us realise that they could use us even more – for the functional genomics schema and the variation schema in particular.

This is what Ensembl is all about. We make it easier for people who want to work genomically to do the sometimes painful data manipulation and plumbing. In particular, Ensembl provides public domain information in a large scale, well organised and ready to be browsed on the web, scripting against in Perl and accessible to clients like bioconductor. And more than any other group, we help group’s like David’s do more for his institute and have to worry less about the infrastructure. David was very interested in the “geek for a week” program when someone comes to work at Ensembl to help accelerate a project.

Returning to the airline theme, some of the biologists admitted using the UCSC browser in a little embarrassed way. I responded that it was fine – UCSC is a great browser, with some great tools. Like airlines, we know people have a choice browsers, and we hope people come “fly ensembl” and enjoy it, but we know the competition is good (and really friendly as well – we like working with those crazy californians, and have a number of joint projects). If you are a biologist, you should use the best tool for the job at hand. Of course, we know where we’re lacking, in particular in comparison to UCSC, and we are working on getting better. Keep an eye open on changes in Ensembl this year – and do come fly with us even if your “regular browser” is US based.

Finally my plane I think is ready to depart.

(Madrid airport is so big I think I’m half way to the UK already)

Today the 1000 Genomes projects was announced. By any measure this is a big deal.
The goal is simple: to create the most comprehensive and medically useful collection of human variation ever assembled by producing approximately 6 terabases of sequence. To put this amount of data in prospective, 6 terabases is more than 60 times the amount of data that is currently available in the DDBJ/GenBank/EMBL Archive and that took more than 25 years to collect. At the peak production of the 1000 Genomes project more that 8 billion basepairs per day will be sequenced. It’s data output of the the entire human genome project every week. All made publicly available.
The data generation rate and the short read length mean that the bioinformatics requires for the project are equally ambitious (or terrifying depending on your point of view). The EBI and NCBI, working together, are creating a joint DCC (data coordination centre) to collect, organise and provide the data to the world. Steve Sherry at the NCBI and I are eager to take this on.
At Ensembl we’ve been expecting this development and built support for re-sequencing data into our variation database a couple of years ago. So far, we have data for about 6 humans, 5 mouse strains, and a smattering of rat data. Small stuff compared to six months from now, but large enough that we have both experience and confidence dealing with the large-scale resequencing data. We are probably going to need both.
Check out more at