Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

local data sources #174

Open
ajo2995 opened this issue Jul 23, 2015 · 15 comments · May be fixed by #175
Open

local data sources #174

ajo2995 opened this issue Jul 23, 2015 · 15 comments · May be fixed by #175

Comments

@ajo2995
Copy link
Contributor

ajo2995 commented Jul 23, 2015

I would like to embed a browser configured with data sources from local javascript objects. Is something like this possible?

var features = [
  {chr: '1', start: 123, end: 234, label: 'foo'},
  {chr: '1', start: 333, end: 444, label: 'bar'},
  {chr: '3', start: 111, end: 222, label: 'baz'}
];
var styles = [
{
    type: 'default', 
    style: {
       glyph: 'PLIMSOLL', 
       HEIGHT: '12', 
       STROKECOLOR: 'black',
       FGCOLOR: 'red'
    }
},
{
    type: 'nasty',
    style: {
        glyph: 'CROSS',
        FGCOLOR: 'green'
    }
}
];
browser.addTier({
    name: 'MyData',
    desc: 'I have these features in memory', 
    localData: features,
    localStyleSheet: styles
});
@ajo2995
Copy link
Contributor Author

ajo2995 commented Jul 27, 2015

After reading some of the code, I decided to modify MemStoreFeatureSource to add an option to bypass the parsing. It will be a less ugly hack if the features can be checked for validity with the source.payload parser. Does this make sense to you? I'll submit a pull request when I get it working.

@ajo2995 ajo2995 linked a pull request Jul 28, 2015 that will close this issue
Open
@max-l
Copy link
Contributor

max-l commented Dec 20, 2016

@ajo2995 is "localData" supported in the current release ?

I tried it and it doesn't seem to work,

it's definitely a worthwhile feature...

@ajo2995
Copy link
Contributor Author

ajo2995 commented Dec 20, 2016

It has not been merged in to the dasmoth/dalliance AFAIK.

I haven't used it yet but the idea was to embed a protein browser in the search.gramene.org results list and display domain annotations as tracks. I already had the positions of the domains on the protein sequence, but I needed to modify things to allow for a protein sequence source adaptor (via ensembl REST) and massage the domain annotations into an acceptable format. I recall getting a small test instance working, but the protein browser feature was postponed.

@dasmoth
Copy link
Owner

dasmoth commented Dec 20, 2016

If you're using the trunk version of Biodalliance, then for small numbers of features, you can use the "test-source" store, as per:

 https://github.com/dasmoth/dalliance/blob/master/test/render-test.html#L146-L151

For a larger number of features, something closer to what @ajo2995 suggests is probably a better bet. Although I'd be curious to know more about the use case -- in general, I've found that making a bigBed file or similar is usually the easiest solution.

@max-l
Copy link
Contributor

max-l commented Dec 20, 2016

@dasmoth the use case I have is that I need to show a small number of trancsripts, proteins, peptides (never more than a dozen at a time), but the data feeds from a database with hundreds of thousands of them, and the small selected subset is deternined by a search result from a database.

Currently I produce a set of huge bed files, and move the view so a specific set of segments can be seen (transcripts, peptides). The problem is that I only want to show a tiny subset of the segmets, and if I load the browser with my bedfiles, everyhing gets displayed (not only the small subset).

If I could "turn off" (erase visually) all the segments in a bed file except for the few desired ones, I wondered at one point if I could achieve this with a rendering callback, it seems not.

One thing I have tried is to create small bed files with only the tiny set I need to show, but this tiny set is the result of a database search (driven by a UI), so it's not so practical to pre generate all of them them (there would be millions), it would be better if I could hand them to the browser in the form of JSON rows from the client in javascript.

A third alternative Is to serve dynamically generated bed files, I have tried that, but it seems the browser doesn't like uncompressed bed files #206 I could convert them to bigbed, but for such small files it's unfortunate if I have to call bedToBigBed...

@moritzschaefer
Copy link
Contributor

moritzschaefer commented Feb 26, 2018 •
edited
Loading

@max-l I have a similar use case. On top it can be useful to compute some features on the fly in the web application and directly show them in a then dalliance sequence browser.
@dasmoth Is there something wrong with the pull request? If so maybe we can edit it so it can get merged.

@dasmoth
Copy link
Owner

dasmoth commented Feb 26, 2018

Sorry, I rather lost track of this one 😊

I'll hopefully have time this weekend to take a look. Moritz -- if you get a chance, splitting the "pre-parsed features" stuff out of the larger pull request would make it that bit easier to consider in isolation.

@moritzschaefer
Copy link
Contributor

I'll try to work on it during the weekend.

@dasmoth
Copy link
Owner

dasmoth commented Mar 3, 2018

Just wanted to point out one other alternative:

    {name:      'Inline BED test',
     tier_type: 'memstore',
     payload:   'bed',
     uri:        URL.createObjectURL(
                     new Blob([
                         ['22 30000000 30010000',
                          '22 30050000 30060000'].join('\n')
                     ])
                 )
     }

(It should be possible to use a data: URL instead of a blob: if that's easier for specific cases...)

This may seem a little more complex, but it does have a couple of advantages:

  • Clear separation between "external" data model's and Biodalliance's internal "feature" type
  • It preserves the possibility of identifying source configs by the URI of the data they point to (this isn't an absolutely hard requirement, but does avoid needing special case code in a few places...)

@TransGirlCodes
Copy link

TransGirlCodes commented Jun 27, 2018 •
edited
Loading

Hi, is it possible to inline something like a BigWig file in this way?

I tried by doing something like this:

        new window.Browser(

{"viewStart":1,"chr":"1","pageName":"39db3c5f-4572-4b5c-954d-5b83c1479504","viewEnd":100000,"sources":[{"bwgURI":URL.createObjectURL(
    new Blob([38,252,143,136,3,0,4,0,200,0,0,0,0,0,0,0,248,0,0,0,0,0,0,0,247,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,160,0,0,0,0,0,0,0,0,64,0,0,0,0,0,0,0,0,0,0,64,0,0,0,0,0,0,0,75,4,0,0,0,0,0,0,58,13,0,0,0,0,0,0,0,1,0,0,0,0,0,0,142,13,0,0,0,0,0,0,169,16,0,0,0,0,0,0,0,4,0,0,0,0,0,0,253,16,0,0,0,0,0,0,234,17,0,0,0,0,0,0,0,16,0,0,0,0,0,0,62,18,0,0,0,0,0,0,144,18,0,0,0,0,0,0,16,39,0,0,0,0,0,0,0,0,0,64,95,232,160,63,0,0,0,96,106,193,239,63,0,0,0,32,164,76,181,64,0,0,0,96,33,228,172,64,145,140,202,120,1,0,0,0,4,0,0,0,8,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,99,104,114,49,0,0,0,0,16,39,0,0,1,0,0,0,0,0,0,0,120,156,61,212,109,72,214,87,28,198,241,171,82,179,238,202,199,212,150,149,101,217,35,98,117,71,82,247,255,252,204,41,40,86,203,205,178,135,27,138,208,117,87,46,163,230,114,180,150,133,65,13,153,173,133,211,205,160,192,182,32,217,12,170,249,194,162,145,140,96,35,238,162,23,22,70,65,110,19,74,10,42,52,87,217,117,21,116,224,251,255,188,59,231,127,14,135,3,188,27,177,51,241,126,12,67,217,91,245,253,174,191,215,201,107,172,242,219,72,147,243,134,1,125,45,125,78,30,101,251,35,22,154,124,198,46,248,31,57,185,118,56,112,124,176,208,100,7,43,217,123,198,201,180,17,192,224,226,42,147,181,44,51,228,60,217,203,202,63,235,117,114,121,4,144,95,243,218,147,109,108,119,77,156,147,137,145,192,177,101,191,122,178,154,237,8,239,114,178,155,253,123,241,186,147,57,81,64,125,184,217,201,22,118,123,81,145,201,232,145,192,39,190,175,77,86,176,35,249,93,78,134,217,245,184,118,39,253,209,192,178,169,89,38,27,217,239,129,59,78,190,100,3,7,190,48,185,105,20,240,127,213,40,147,157,172,234,82,208,228,172,209,220,207,151,255,56,89,199,134,18,154,60,249,132,53,108,29,107,178,196,7,252,212,176,196,100,59,219,21,179,215,100,234,24,160,245,210,58,147,53,236,191,246,67,78,246,176,194,79,203,77,22,140,5,238,23,23,153,108,101,253,87,103,120,50,118,28,231,133,223,228,231,172,35,242,27,39,187,88,209,221,225,38,3,49,60,19,31,76,158,100,115,134,54,152,140,136,229,89,108,46,55,25,98,73,63,223,241,228,223,236,214,160,223,100,86,28,176,101,230,95,158,60,206,6,238,254,226,222,202,158,215,79,49,25,140,7,210,123,146,77,94,97,147,138,47,59,57,61,1,248,51,144,96,242,48,91,121,122,141,201,71,108,107,242,31,78,174,74,228,255,79,107,115,242,60,75,109,250,216,100,202,120,174,151,249,145,201,175,152,175,116,188,201,251,204,59,183,192,100,94,18,240,192,95,104,242,12,155,29,222,237,228,152,100,32,183,62,206,228,78,150,152,189,218,228,45,54,180,237,132,147,217,41,192,252,123,48,217,204,126,204,141,50,137,9,192,247,17,27,77,150,177,156,135,158,201,107,108,114,203,99,39,231,125,0,100,60,244,153,60,202,178,87,31,243,228,51,54,152,59,210,147,107,39,2,81,129,60,147,29,172,237,212,13,79,166,165,2,167,75,107,157,172,101,241,161,124,147,189,236,172,183,198,228,242,73,188,255,157,75,77,182,177,248,210,113,38,19,39,3,77,121,15,156,172,102,87,19,206,57,217,205,214,151,101,120,50,103,10,176,47,101,187,201,22,246,234,183,128,201,232,52,96,221,193,185,38,43,88,168,127,132,201,48,11,102,151,155,244,79,5,138,183,103,152,108,100,231,251,110,59,249,146,69,103,109,49,185,105,26,112,51,152,110,178,147,173,218,184,194,228,172,116,96,113,245,82,147,117,44,233,195,37,38,159,176,230,31,54,155,44,153,14,44,218,22,48,217,206,130,251,187,156,76,157,193,187,88,176,207,100,13,171,124,58,224,100,15,171,235,223,227,100,65,6,247,83,241,194,147,173,236,72,76,165,147,122,183,66,37,153,246,6,151,130,18,234,224,172,104,36,64,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,16,39,0,0,247,3,0,0,0,0,0,0,1,0,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,16,39,0,0,0,1,0,0,0,0,0,0,247,2,0,0,0,0,0,0,157,0,0,0,120,156,77,216,123,116,79,87,22,7,240,131,68,164,241,74,60,18,84,144,16,163,49,94,9,34,242,187,231,119,246,208,241,174,72,48,17,197,32,146,122,20,177,48,168,184,53,177,198,107,36,69,169,16,109,103,121,140,25,212,35,245,168,182,110,4,107,97,22,65,25,209,210,26,207,241,156,97,76,194,132,204,217,103,239,174,28,107,125,255,250,220,155,239,239,236,115,239,253,221,31,33,232,159,159,243,81,249,61,135,163,182,213,72,85,63,155,107,249,244,85,129,242,230,221,114,213,179,203,76,227,104,30,59,26,7,10,71,173,48,142,38,106,144,63,218,242,200,156,255,32,229,223,42,252,250,4,227,198,44,95,20,16,39,155,253,41,24,74,194,10,168,95,155,203,142,198,129,53,99,94,82,191,54,143,253,203,248,135,230,252,91,169,129,208,124,88,62,245,107,19,53,171,125,237,203,254,50,227,187,64,24,48,104,45,245,163,177,163,113,32,59,189,21,152,126,109,46,123,234,130,63,155,243,23,79,20,224,196,210,231,67,243,44,127,217,115,182,220,92,240,181,10,216,21,69,253,218,68,45,114,52,14,116,189,53,217,252,125,99,236,157,178,28,31,186,147,39,160,116,230,3,90,191,54,215,114,142,255,220,230,122,142,233,215,230,89,158,49,237,158,51,118,194,247,106,238,139,190,212,175,77,4,144,247,117,95,27,159,212,122,135,106,113,33,152,214,143,102,57,199,127,170,230,66,105,250,181,185,150,207,114,67,157,137,141,235,171,163,205,55,27,71,243,216,87,171,221,198,151,222,171,169,78,255,184,220,56,154,8,172,118,142,63,51,241,145,249,252,198,44,127,191,52,219,89,209,1,84,86,65,123,191,233,215,230,178,163,221,57,112,214,169,23,150,171,58,44,92,101,28,205,99,71,227,168,107,247,99,104,253,218,68,109,242,188,210,77,198,135,111,221,167,226,35,95,155,243,141,89,206,81,123,90,122,212,175,205,101,47,235,62,80,114,96,230,144,54,102,255,208,60,203,83,66,114,164,51,235,183,80,20,212,223,56,154,8,34,71,227,192,188,230,31,208,254,163,177,47,235,123,197,65,223,117,180,82,189,122,190,138,246,95,155,203,126,54,244,144,131,199,140,233,126,86,29,174,160,245,161,121,150,115,212,134,180,0,90,191,54,81,167,218,85,155,46,50,182,69,11,56,247,206,73,218,127,52,246,131,73,87,141,31,200,143,134,204,244,203,212,175,205,181,156,163,230,132,36,82,191,54,207,242,138,15,231,200,194,74,128,171,103,218,209,250,181,137,96,242,255,205,14,150,232,9,123,167,67,84,187,9,180,126,52,203,57,240,114,243,26,234,215,230,90,62,251,155,209,242,80,179,56,184,20,22,64,243,215,230,177,163,113,224,151,35,28,234,215,38,222,32,207,157,119,219,65,223,120,248,169,170,49,109,6,173,31,141,189,170,209,6,31,30,115,225,68,161,42,43,170,73,253,218,92,203,57,254,243,161,7,233,254,211,230,89,190,238,189,122,50,183,226,161,154,242,52,135,230,175,77,132,144,163,109,92,151,40,255,94,20,1,23,98,15,83,63,26,59,26,7,186,85,5,209,243,71,155,107,121,118,131,5,114,247,63,39,193,177,19,41,180,126,109,30,251,206,111,210,140,39,222,158,3,199,215,189,79,235,215,38,234,86,59,7,154,199,245,162,249,163,177,223,61,180,196,65,175,188,30,0,143,247,126,73,235,215,230,90,206,81,3,247,175,48,247,7,154,199,222,127,82,134,228,64,251,252,193,212,175,77,212,35,255,41,121,160,241,244,87,105,240,211,219,137,212,143,102,57,7,158,205,167,251,11,205,101,47,47,105,231,67,63,251,32,88,13,153,188,133,250,181,121,150,167,138,120,25,145,179,80,21,167,183,162,249,107,19,245,249,249,172,141,3,15,91,191,162,249,163,177,127,29,184,220,65,159,155,91,172,154,141,235,65,235,215,230,90,62,240,90,77,185,181,231,231,234,111,125,174,81,191,54,143,29,141,3,85,167,151,80,191,54,209,128,159,47,33,194,120,143,230,53,96,77,226,98,234,71,179,252,173,170,116,121,234,116,32,44,58,152,71,253,218,92,118,52,14,228,215,247,209,254,107,243,44,159,58,62,67,158,217,51,14,238,12,236,75,243,215,38,26,146,163,113,160,247,50,218,31,99,236,77,183,93,245,161,207,185,220,74,53,28,50,207,172,15,205,181,252,187,151,241,50,236,206,14,85,163,199,175,233,254,215,230,177,163,113,96,220,205,42,90,191,54,17,74,158,217,254,140,57,191,77,159,81,42,118,64,169,249,251,198,44,175,184,182,221,57,214,178,181,58,54,184,46,245,107,115,217,209,56,170,40,230,7,154,191,54,207,242,231,121,173,228,246,71,65,112,240,233,6,234,215,38,194,200,163,111,133,75,244,249,5,145,240,98,245,113,154,63,154,229,28,40,243,21,209,252,181,185,236,45,147,191,117,208,253,207,159,168,152,65,211,104,253,218,60,203,57,234,195,15,234,81,191,54,209,136,252,100,82,35,201,129,162,119,119,80,63,154,229,67,182,142,144,139,191,234,2,25,223,86,82,191,54,151,29,141,3,167,42,226,104,255,181,121,236,239,133,23,59,232,165,133,215,213,47,166,166,81,191,54,209,152,175,255,168,61,14,30,115,225,244,17,117,176,138,191,255,209,44,231,168,25,99,158,210,252,181,185,150,191,185,97,152,108,93,30,7,155,23,7,83,191,54,143,125,109,167,119,36,122,244,243,100,248,79,92,71,186,254,180,137,38,213,206,129,209,59,219,211,245,135,198,30,50,178,137,241,235,65,177,48,108,251,125,90,191,54,215,114,223,222,110,114,119,199,112,152,249,175,125,52,127,109,30,59,26,7,238,206,126,65,235,111,130,23,110,181,223,140,239,47,27,62,243,193,146,233,245,168,31,141,29,141,3,225,109,35,233,249,163,205,101,239,80,58,203,65,239,60,99,133,42,189,127,132,174,63,109,158,229,144,23,42,115,70,31,86,219,167,181,166,126,109,34,156,28,141,3,221,254,176,149,230,143,102,121,227,132,225,114,234,210,126,80,118,150,223,63,181,185,236,85,147,11,29,244,233,235,227,224,143,190,16,154,191,54,207,114,142,186,248,73,177,249,124,104,34,162,218,187,254,40,100,202,196,127,168,163,219,135,82,63,26,59,90,1,212,150,155,87,212,128,119,163,126,79,243,215,230,178,163,113,224,227,139,43,105,254,218,60,203,215,4,140,149,7,58,39,64,106,120,93,218,127,109,162,25,57,26,7,42,250,1,205,31,141,221,255,192,39,57,208,253,54,157,143,230,178,71,110,121,226,160,39,250,91,65,252,218,115,212,175,205,179,156,163,202,82,198,209,252,181,137,230,213,30,243,32,68,214,202,10,134,223,148,209,251,133,49,246,132,225,171,125,232,149,231,207,168,101,79,82,104,253,218,92,203,57,254,207,6,108,167,239,127,109,30,251,75,8,50,126,101,83,39,127,132,12,54,142,38,90,84,123,237,164,62,242,211,109,157,212,7,217,55,205,254,24,99,71,227,64,249,148,166,180,126,109,46,251,158,207,207,155,243,251,61,222,175,242,71,100,209,250,181,121,150,111,29,153,235,228,20,222,246,39,167,101,208,251,183,54,241,38,57,26,71,137,75,75,169,31,205,242,176,172,190,242,233,169,90,240,223,20,122,62,161,185,236,104,28,24,209,43,130,174,63,109,158,229,127,245,141,144,201,93,134,67,72,65,87,218,127,109,162,37,121,223,227,189,141,183,27,58,8,34,138,162,105,255,209,44,231,64,202,33,122,190,160,185,236,97,35,235,27,95,249,125,83,56,233,167,231,51,154,199,190,161,207,77,7,143,185,209,33,8,50,159,208,247,39,154,136,172,118,142,250,234,88,127,218,127,52,246,146,70,123,141,215,46,241,84,42,52,162,245,107,115,217,71,77,140,241,225,49,63,44,47,84,71,78,92,54,243,67,243,44,231,248,63,25,116,197,103,250,181,137,86,213,190,48,98,138,204,61,93,7,210,212,51,234,71,99,71,227,64,197,237,145,180,126,109,46,251,171,47,146,36,7,158,135,241,253,175,205,99,79,91,28,107,188,118,199,238,112,103,145,160,249,107,19,173,171,157,3,103,206,222,166,126,52,246,172,242,90,198,79,222,171,3,35,79,242,251,183,54,215,242,209,9,25,178,40,61,10,114,74,175,209,252,181,121,236,104,28,184,225,227,247,63,109,162,13,121,242,148,24,227,107,35,123,64,101,54,189,191,27,99,47,122,84,230,224,49,11,182,182,128,73,93,139,169,95,155,107,57,71,93,201,237,69,253,218,60,203,235,116,201,148,235,71,245,130,183,111,52,162,126,109,34,138,252,194,232,104,137,126,233,233,120,232,42,7,80,63,154,229,28,248,75,78,41,245,107,115,45,31,58,118,176,12,29,147,0,249,110,109,154,191,54,143,29,141,3,167,139,219,82,191,54,17,77,222,243,119,189,141,239,88,151,4,89,130,246,207,24,123,211,95,37,74,60,102,64,64,111,40,252,180,14,237,191,54,215,114,14,172,90,66,191,15,208,60,203,55,173,31,47,75,130,210,33,108,76,18,245,107,19,109,201,187,79,78,50,14,117,211,32,44,156,223,255,209,44,231,192,91,59,223,160,126,109,46,251,232,69,87,28,244,138,65,129,240,209,198,143,105,254,218,60,203,3,250,45,148,51,139,43,213,235,163,244,251,21,77,180,35,71,227,192,212,245,243,169,31,141,125,250,179,10,115,254,139,139,241,176,127,87,3,234,215,230,90,206,81,121,17,217,212,175,205,99,95,89,62,215,225,168,207,58,207,240,155,126,109,34,134,124,203,212,23,62,244,90,33,160,250,125,209,198,184,49,203,57,254,37,1,179,232,255,63,180,185,150,47,107,48,221,25,127,124,156,202,124,60,192,156,143,230,177,163,101,165,118,146,17,81,71,85,66,9,253,62,70,19,237,249,254,209,198,129,125,211,30,209,253,135,214,222,188,134,219,174,180,251,255,15,171,153,181,186,224,172,104,36,64,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,64,39,0,0,58,13,0,0,0,0,0,0,1,0,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,64,39,0,0,79,4,0,0,0,0,0,0,235,8,0,0,0,0,0,0,39,0,0,0,120,156,69,210,105,108,140,81,20,6,224,15,221,40,186,153,182,150,106,171,106,82,101,180,73,169,14,223,61,51,247,86,168,218,74,205,88,6,9,29,26,42,182,148,84,83,91,74,144,210,160,150,137,109,136,45,72,137,84,53,177,38,150,254,104,98,233,31,148,80,90,75,162,182,8,83,147,216,206,188,19,189,201,205,253,241,228,61,39,57,247,104,154,255,116,194,221,229,121,47,150,237,12,164,176,124,143,156,240,117,153,252,111,157,241,94,206,104,19,85,222,28,106,255,22,161,62,169,106,56,91,23,120,126,233,25,225,205,44,166,216,18,163,74,212,52,229,115,182,0,184,169,80,232,236,141,179,79,203,162,172,241,200,179,5,194,119,91,171,245,85,27,34,68,91,245,55,107,182,115,133,197,231,108,65,29,254,182,246,190,184,237,118,201,61,149,13,86,159,179,5,195,43,31,30,18,211,66,215,209,144,177,66,25,14,199,163,63,91,8,252,126,68,157,207,39,76,254,36,215,174,41,70,127,182,174,240,43,163,155,68,251,198,213,100,126,156,166,10,206,132,32,207,214,13,94,94,242,70,20,95,119,80,217,217,129,170,57,165,21,121,182,80,248,159,40,151,126,112,159,153,172,7,235,228,139,165,2,206,214,29,206,182,50,172,148,26,182,58,84,77,139,5,245,217,122,192,223,213,109,22,57,11,157,116,183,62,81,101,68,121,145,103,235,9,247,220,78,214,155,243,114,201,190,239,139,108,204,244,207,159,45,12,126,45,112,187,200,215,50,168,254,108,147,92,172,38,194,217,194,253,243,9,213,168,104,190,147,114,61,57,234,249,231,100,244,103,139,128,47,50,54,232,236,86,115,141,180,205,153,141,60,91,100,135,127,175,140,167,177,244,72,110,175,152,6,103,139,130,199,229,221,16,73,173,49,20,180,75,83,237,55,183,192,217,122,249,247,99,192,69,49,233,164,141,66,146,13,106,173,235,46,156,205,208,225,253,92,83,73,203,77,85,115,31,255,130,179,69,195,67,237,6,170,50,77,166,17,181,67,148,169,180,13,206,22,3,79,121,184,74,180,100,228,144,190,247,183,52,91,14,192,217,98,253,255,179,248,176,232,53,114,58,189,234,20,166,234,106,174,194,217,122,195,211,95,106,180,39,96,30,185,111,25,213,241,161,31,224,108,125,224,253,79,124,22,150,15,58,45,112,197,168,146,25,254,60,91,95,184,87,6,235,65,163,179,169,79,189,73,118,47,63,133,253,100,235,7,191,120,236,145,207,109,219,110,72,119,186,19,121,182,56,248,73,123,185,56,167,219,232,200,121,179,154,83,17,142,255,97,235,15,119,101,183,136,49,119,70,209,159,166,72,85,145,112,1,121,182,120,248,172,130,65,122,89,236,18,122,145,122,76,38,212,102,193,217,18,224,51,55,165,18,123,109,156,84,61,15,24,80,159,45,17,94,232,233,66,142,145,78,122,118,42,93,69,30,237,12,103,27,0,175,249,248,84,132,164,45,162,229,230,20,149,182,227,7,234,179,37,193,27,29,73,52,101,222,68,202,44,202,82,91,220,193,200,179,13,132,71,43,51,29,218,63,159,154,108,227,85,213,194,68,56,91,50,220,177,254,137,8,24,87,70,94,251,48,101,127,30,8,103,27,4,63,81,244,83,103,31,252,238,129,188,151,231,159,31,155,177,195,11,243,77,20,126,252,146,116,191,30,14,255,103,22,35,214,156,205,127,173,151,150,126,180,252,5,243,184,9,13,224,172,104,36,64,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,64,39,0,0,169,16,0,0,0,0,0,0,1,0,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,64,39,0,0,146,13,0,0,0,0,0,0,23,3,0,0,0,0,0,0,9,0,0,0,120,156,99,96,128,2,22,8,214,203,176,179,253,101,94,106,239,96,200,228,146,114,109,145,51,92,142,3,66,79,114,92,103,27,204,93,103,223,25,117,193,185,202,168,10,34,15,146,227,129,200,255,23,158,105,91,196,95,109,207,57,77,217,197,97,222,87,136,60,72,78,0,34,255,253,176,170,173,103,90,170,253,139,112,17,23,217,205,215,32,242,32,57,17,136,124,186,250,105,219,156,164,84,251,76,203,55,206,51,62,119,64,228,65,114,18,16,121,205,11,197,118,34,22,161,246,185,147,37,93,30,76,60,15,145,7,201,201,64,228,127,57,177,219,78,102,137,183,183,210,120,234,252,103,253,4,136,60,72,78,1,34,31,149,162,102,91,43,145,109,127,129,79,194,69,86,16,106,63,72,78,5,34,191,229,237,77,59,14,131,116,123,137,149,230,46,134,187,57,93,192,242,64,57,7,117,160,51,153,25,24,150,228,252,180,101,241,168,181,127,127,96,175,243,61,145,18,103,0,146,176,66,33,224,172,104,36,64,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,64,39,0,0,234,17,0,0,0,0,0,0,1,0,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,64,39,0,0,1,17,0,0,0,0,0,0,233,0,0,0,0,0,0,0,2,0,0,0,120,156,99,96,128,2,1,8,254,126,88,213,54,152,187,206,94,118,50,135,171,189,192,110,23,184,156,2,132,254,229,196,110,91,43,145,109,127,100,10,155,235,146,139,235,33,242,64,57,7,117,160,52,59,3,195,146,156,159,182,44,30,181,246,39,92,186,93,50,14,184,185,0,0,9,201,20,38,224,172,104,36,64,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,64,39,0,0,144,18,0,0,0,0,0,0,1,0,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,64,39,0,0,66,18,0,0,0,0,0,0,78,0,0,0,0,0,0,0])
)
,"name":"User Data"},{"name":"Genome","twoBitURI":"http://www.biodalliance.org/datasets/hg38.2bit","tier_type":"sequence"}],"coordSystem":{"taxon":9606,"ucscName":"hg38","speciesName":"Human","auth":"GRCh","version":"38"}}
        );

Where the array is an inline BigWig file, but it tells me the magic number is incorrect (which I've checked and in the above array the magic number is actually correct (0x888FFC26) according to my hexdump:

N82106:GenomeBrowsers bward$ hexdump mybigwig.bw
0000000 26 fc 8f 88

Is it possible to inline BigBed and BigWig in a similar way to the Inline BED test as above?

@dasmoth
Copy link
Owner

dasmoth commented Jun 27, 2018

The reason you're getting a "bad magic" error is that the Blob constructor doesn't treat arrays of ints as bytes in the way you might hope -- It's actually converting them to strings!

The solution to this is to do:

          new Blob([new Uint8Array([38, 252, 143, 136, ...])])

There are potentially a couple of other issues here. The first is that Biodalliance needs random access to the file, which won't necessarily work with blob: URLs. There's actually an easy way around this:

           sources: [
                {
                      name: 'User Data',
                      bwgBlob: new Blob(...)
                }
          ]

This was originally an internal mechanism which is used when you create a track from a local file (files appear inside the browser as Blobs), but does work with arbitrary Blobs, so I'll add this to the docs next time I'm in the area.

This got your example mostly working for me. I did run into a problem when browsing beyond the end of the data. I need to look into this more closely -- it might imply a truncated bigWig file, but it's not impossible that it could be a Biodalliance error. I'll investigate further...

@dasmoth
Copy link
Owner

dasmoth commented Jun 27, 2018

Okay, there was a boundary-condition bug which could affect a few very small bigwig files (including yours). Fixed now.

@TransGirlCodes
Copy link

TransGirlCodes commented Jun 27, 2018 •
edited
Loading

Wow that's brilliant, thank you.

The reason I find the ability to inline some of these things desirable, is because I'm building a package https://github.com/BioJulia/GenomeBrowsers.jl/tree/updates, so our users can see their results in Dalliance in Jupyter.

By default a local https server gets spun up in the background, keeping the data, like the array above in memory, and serves it to the Dalliance instance in the notebook. This is fine until you want to look at the notebook in a viewer, or if the notebook is converted to reveal.js slides.

Then you need to either, recreate the server that got spun up (my expeirence with fellow biologists with no interest in computers is they won't bother), or have dalliance read data from local files, or inline all the data.

@dasmoth
Copy link
Owner

dasmoth commented Jun 27, 2018

Thanks Ben, sounds like a great project, and some extra encouragement for me to give Julia a try one of these days.

If there's anything we can do Biodalliance-side to better support this sort of use case, please do shout. I certainly agree that avoiding server restarts and the like is very worthwhile. For more general discussion about this kind of thing, the mailing list ([email protected]) might be a better format.

@TransGirlCodes
Copy link

TransGirlCodes commented Jun 29, 2018 •
edited
Loading

Thanks @dasmoth, one of the things I'm curious of is support for custom genomes.
I work at an institute that does genomic sequencing and research of some interesting organisms.

Say I have a FASTA file of some reference genome, and then some BAMs, some SAMs and so on, and I want to visualise them, I know I can enter the BAM, SAM and other track datasets.

I know that Dalliance will accept a TwoBit URI or Blob for the actual sequence itself, but I was wondering about the other parameters like: "coordSystem":{"taxon":9606,"ucscName":"hg38","speciesName":"Human","auth":"GRCh","version":"38"}. Sometimes genomes we sequence and species we work on might not nessecerily have a version, a ucscName or taxon code if it's not a model organism that has been worked on a lot or isn't on the DAS registry and other hubs and databases and so on. Would this be an issue for someone who just wanted to load their FASTA reference sequence and some alignment / annotation files?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

pre-parsed features and other things I like ajo2995/dalliance
5 participants
@max-l @dasmoth @ajo2995 @moritzschaefer @TransGirlCodes