Hot on the heels of release 51 comes release 52 of Ensembl – the first revision of the new webcode… So what’s new?

Data:

Web site:

  • Updated export: – Restored most of the functionality with the new Export wizard on Genes, Transcripts and Locations – to allow export of FASTA, EMBL, Genbank, GFF, TSV, Vista and PIP files.
  • Image export: – Restored an improved version of the image export functionality – all “Horizontal” generated images have and [Export image] button to allow the image to be exported in vector format (PDF, SVG, EPS) and scaled bitmap format (PNG x0.5, x1, x2, x5 and x10) to allow publication quality images to be exported.

    The vector formats PDF, SVG and EPS can all be imported into vector image editors to be manipulated as well.

We’re happy to announce that Ensembl is one of the launch partners for Amazon’s “Public Data Sets” initiative, so the MySQL data and index files for the current release of Ensembl can be accessed from within Amazon’s Elastic Compute Cloud (EC2) service. From the Amazon website:

AWS Hosted Public Data Sets provide a convenient way to share, access, and use public domain or non-proprietary data within your Amazon EC2 environment. Select public data sets are hosted on AWS for free as an Amazon EBS snapshot. Any Amazon EC2 customer can access this data by creating their own personal Amazon EBS volume from a publicly shared Amazon EBS public data set snapshot. They can then access, modify, and perform computation on these data sets directly using an Amazon EC2 instance and just pay for the compute and storage resources that they use.

Details of how to access the data can be found at http://aws.amazon.com/publicdatasets .

We have plans to make much more use of AWS in the future, stay tuned!

Due to the changes in the web interface there have been a number of changes to the URLs for pages. In most cases the web code catches these changes but there are a number of requests which due to the nature of the site have changed:

  • Configuring the way a page is rendered;
  • Changing the way tracks are rendered;
  • Adding DAS sources via a web-address and not via the web interface;
  • Attach UCSC style external resources.

These are now all attached in a similar – systematic way:

  • To change global page settings: add a paramter config=key=value{,key=val}
    e.g.
    to turn off the top image on Location > Region in detailhttp://www.ensembl.org/Homo_sapiens/Location/View?r=1:1000-2000;config=view_top=off

    e.g. to link directly to the Exon Intron markup panel (Transcript > Exons) and to show full introns and only 60bp flanking sequence AND turn the display to be 60bp wide

    http://www.ensembl.org/Homo_sapiens/Transcript/Exons?t=ENST00000309255;config=flanking=60,seq_cols=60,fullseq=yes

  • To change configuration for an individual panel add a parameter refering to the panel (this will be documented shortly on the website) e.g. For Location > Region in detail the two panels are contigviewtopcontigviewbottom, for Location > Region overview it is cytoview. This is again a comma separated list, where the left hand side of each “=” is the name of the track, and the right hand side is the name of the “renderer” to use – the latter depends on the type of track. Additionally the left hand side can be used to integrate external data: Notes:
    • Track names are now systematically named so will have changed from the values you may have been used to using – again we will shortly publish a list of these, but examples are: transcript_core_ensembl – the ensembl genes from the ensembl database.
    • Renderers depend on the type of track, but e.g. for transcripts you have the option of “transcript_label”, “transcript_nolabel”, “collapsed_label” and “collapsed_nolabel”, for alignment features (and also url attached data at the moment) “normal”, “half_height”, “stack”, “unlimited” and “ungrouped”, for DAS tracks “labels” (show labels if configured by the source) or “nolabels” – hide labels.
    • At the moment two special parameters can be used:
      das:http://www.mydas.source/das/my_data=render
      – which attaches a DAS source to the session and selects the renderer
      url:http://www.myweb.server/my_data.format=render

    For example:

    http://www.ensembl.org/Homo_sapiens/Location/View?g=ENSG00000012048;config=panel_top=off;contigviewbottom=das:http://www.ensembl.org/das/Homo_sapiens.NCBI36.transcript=nolabels,transcript_core_ensembl=collapsed_nolabel

    Turns on a das source (in this case the Ensembl transcripts) and collapses the standard ensembl track down to a single line per Gene AND also turns off the top panel!

The web team can finally let out a quick sigh of relief now that the long awaited new web code has finally emerged kicking and screaming out of the web team office…

It is obvious to see the “cosmetic” changes to the site:

  • the colours,
  • fonts,
  • layout,
  • the unified configuration
  • the reduction in page sizes.

On top of this there have been a large number of underlying technical improvements to the way the pages are put together.

  • Streamlining the JavaScript and css to make sure that the transfers to and from the server to your browser are as fast as possible; Using unobtrusive JavaScript throughout the new code so pages work with or without JavaScript or AJAX – althouth they are not quite as functional they still work!
  • Making the pages standards compliant to make them render in most browsers without issues (unless of course that browser is IE and there are lots of places where the “standards” approach fails)
  • Using an fast in memory cache (a modified version of memcached which allows for the use of tags) to reduce the load on our user database and to store and server temporary images, processed HTML etc.
  • Segregation of code into more modules to reduce the size of the very large modules we had (noticeably the breakdown of the Component modules into smaller chunks)
  • Configuration meta information contained in core databases making the site easier and more automatic to set up.
  • Optimisation of drawing and configuration code.
  • Transparent use of AJAX in many cases. Use of Perl’s LWP::ParallelUserAgent where the user’s browser doesn’t support AJAX.
  • Further areas where the extensible plugin system is available – defining colours, configuring images.

There are still a few more Ensembl training events before the end of the year.

Browser workshops:

UNAM, Mexico City, Mexico (1-2 Dec)
UNAM, Cuernavaca, Mexico (5 Dec) (+ departmental seminar 4 Dec)

Amsterdam, The Netherlands (19 Dec)

Developers workshop:

University of Cambridge, UK (1-3 Dec)

In addition, Ensembl will feature as part of the following courses:

Wellcome Trust Open Door Workshop ‘Working with the Human Genome Sequence’ (1-2 Dec, Hinxton, Cambridge, UK) and Genes en evolución, ecologia e conservación (8-9 Dec, La Paz, Baja California, Mexico)

For details of these workshops, please have a look at the complete list of Ensembl training events.

Do you know a bit of Perl? Ensembl hosts an API (Application Programmers Interface) which uses Object-Oriented Perl to extract data from Ensembl databases. This API is public and can be used for people to programmatically access the data in the Ensembl database. We understand that not everyone is used to Object-Oriented code, although people may have basic Perl skills and be interested in using our datasets. For that kind of bioinformaticist, I would recommend a recent short read in O’Reilly’s Broadcast:

Beginners Introduction to Object-Oriented Programming with Perl – O’Reilly Broadcast

And for the more advanced readers, the classic reference book in OO-Perl would be Damian Conway’s Object Oriented Perl, which a part from being very informative, has a really cool cover 🙂

We are always trying to lower the barrier to entry for research communities interested in using the Ensembl database in programmatic ways that make use of all the complexity associated with the generation of our data. That’s why our API is public and well-documented. You can learn about our API by attending on of our API workshops for free (e.g.: 1-3 December – Univ. Cambridge, UK). We are currently trying to smooth things out even more, working on ways to make it even easier to download all that’s needed to use the API and have the example scripts running in your computer with the minimum number of steps. Keep tuned for news in this respect soon…

Ensembl is attending the 22nd International Mammalian Genome Conference taking place in Prague (Czech Republic) from 2-5 November 2008. This meeting starts with three bioinformatics workshops on Sunday (2nd November) at the Institute of Molecular Genetics (AS CR). One of these workshops will focus on Ensembl, discussing new developments and featuring a preview of our new interface. We will be starting a 9:00 (Seminar Room 3.102, in the third floor). You can download the workshop materials here (exercises and tutorials). As part of our commitment with the EURATools consortium, we’ll be focusing on rat genomics, but if you work with any other species annotated in Ensembl, you are welcome.
See you in Prague!

Only 3 continents to cover this time, but November will be even busier for the Ensembl trainers than October ….

Ensembl will feature as part of the following courses:

‘Computational & Comparative Genomics’ (5-11 Nov, Cold Spring Harbor Laboratory, New York, US)
Wellcome Trust Open Door Workshop ‘Working with the Human Genome Sequence (10-12 Nov, Wellcome Trust Genome Campus, Hinxton, UK)
Hands-on training at EBI ‘Programmatic access in Java: webservices & work flows’ (24-27 Nov, Wellcome Trust Genome Campus, Hinxton, UK)

Browser workshops will be given at the following locations:

Europe:
Prague, Czech Republic (2 Nov)
Madrid, Spain (5 Nov)
Newcastle, UK (13 Nov)
Cambridge, UK (13-14 Nov)
Naples, Italy (19 Nov)

North America:
Cambridge, Massachusetts, US (12 Nov & 14 Nov)
Boston, Massachusetts, US (13 Nov)

Asia:
Kuala Lumpur, Malaysia (24-25 Nov)
Sabah, Malaysia (27-28 Nov)

For details of these workshops, please have a look at the complete list of Ensembl training events.

Ensembl is currently down due to a power outage in the data centre at the Sanger Institute last night. Power has been restored, but it will take some time to restore all of the services.

We are working to get things up and running and expect that Ensembl will be back mid to late morning UK time.

New design
You will already have seen a number of emails about the upcoming Ensembl 51 release – the web team are working hard to tidy up the loose ends of the release! We have got most of the major views ready, and just working on some of the views you may have never found before. As a taster I’m posting a few screen shots from our development site, the first shows the new page layout for graphical display of genomic regions (the old contigview). You will see many of the new design decisions in this screen shot:

  • There are more views per object as we have broken up the large single pages into smaller components;
  • Tabs for the different focus objects – in this case Gene and Location. Transcript and Variation feature are the other tabs available;
  • A tree of all information available about the focus feature on the left hand side;
  • Left/right pagination buttons to allow you to navigate between all the information we have about the focus object.
  • “General” and “local” tools areas

Under the hood!

There have been a large number of changes under the hood of the web-site. Notable changes have been:

  • Use of modified version of memcached to store and retrieve cached images, static and dynamic content, user settings;
  • Re-writing the configuration code to automagically detect the contents of the databases and try and display the content appropriately;
  • Breaking up of the component code into separate modules;
  • Removing the need for a script per view – by using “routeing” style URL parsing to work out what objects are to be rendered and how… e.g. /Gene/Compara_Tree/Text displays the text version of a gene’s homology tree.
  • More and easier to configure renderers for drawing code.
  • A strive for standards compliance in both XHTML and CSS; which should allow us to support more easily modern web browsers. We will be actively supporting Firefox 3+, Internet Explorer 7+ and Safari 3+ (and other similar browsers), while trying to make sure that the site is still workable in other browsers (at the site appears to work in Opera 9.25+)

New configuration panel

All configuration of the site and individual views has been moved to a common “Configuration dialog” box.

  • The old “yellow menus” are replaced by a more expansive and easier to navigate tree of features. Important now there are nearly 200 individual tracks in the Human Location view page.
  • There are more choices to display some tracks – rather than just turning them on and off, you can decide how you wish them to be displayed.
  • Configuration for other pages are loaded in a similar way.
  • The site has a common site-wide image width setting.
  • The configuration panel is also where you will: manage your accounts, upload data, attach DAS and URL based data

Different renderers

For different data types we now support different renderers – not just collapsed and expanded.
For example:

  • For genomic alignments we support, the ungrouped features (all on one line), normal grouped and bumped features at both full and half-height, and now also “stacked” features – “2 pixel” high glyphs.

We hope when you see the new interface that you will find it more intuitive, more discoverable and faster to use and most importantly more productive for the research work that you are doing.