Tiny time differences: Intentional, or safe to round?

[clawsoon]

Hi,

I'm doing some visualizations of data from your older "Inferring whole-genome histories" paper (and hoping to run the same code on the "A unified genealogy" data when it's publicly released).

There are a bunch of times in the treeseq data which look something like 4.999999999999343. Am I safe to assume that means 5? Or is it intentionally "five minus epsilon", close to 5 but not quite there?

I'm guessing all the 4.999... stuff is just floating point math being its occasionally irritating self, but I figured I'd double check.

Thanks.

[hyanwong]

Hi @clawsoon - I suspect that (a) you are using an older version of tsinfer, where most times are integers (in more recent versions nodes are placed at frequencies of 0..1, apart from the 2 oldest nodes at times 2 and 3 which I hope to change soon ) and (b) these are "synthetic ancestors" created by the path compression process (you can check by looking to see if they have the flag tsinfer.NODE_IS_PC_ANCESTOR).

Synthetic ancestors are intentionally placed at a time X-minus-epsilon: you can't place these at time (say) 5, because their parent is at time 5, and children must always be strictly younger than their parents. So it's not just floating point error, but a deliberate choice. The time value 5 (or 5/N in more recent tsinfer versions) simply means that 5 samples possess the focal allele on which the ancestor is based. Since synthetic ancestors don't have a focal allele, there's no obvious time to assign to them, so they just get placed a little below their known parent node.

However, I should emphasise that the absolute node times in a tree sequence output by tsinfer are fairly meaningless (although their relative order is, of course, important). So you could swap all the values of 4.999999999999343 for 5 and increase all the nodes at time 5 and above by 1, and the tree sequence should be just as meaningful. If you want to use the node times in any meaningful way, then you should run tsdate on your tree sequence first, which will get rid of those 4.999999999999343 values anyway.

[clawsoon]

Thanks for the detailed explanation, @hyanwong. You are correct - looks like every one of them has tsinfer.NODE_IS_PC_ANCESTOR set.

I'm not actually using tsinfer myself, but instead using the published tree sequences from here and here. I tried running tsdate myself, but quickly ran out of memory, so I'm eagerly awaiting the publication of dated tree sequences from your group.

[hyanwong]

Right - the published tree sequences use the older tsinfer version. You might be able to run tsdate on the published tree sequences if you want a (rough) time estimate for the nodes - I'm surprised that you ran out of memory, actually. Is there any reason why you need the node times rather than simply the topology?

[clawsoon]

I ran out of memory because my Linux machine only has 1GB of RAM. :-) I'm making a very long movie by plotting the current geographic locations of every mutation on chromosome 22, sorted by date. I'm hoping to release it publicly when it's done, so I'd feel more comfortable if the rough dates were generated by someone who actually knows what they're doing rather than by me. :-)

If your group isn't planning to release a dated tree sequence for chromosome 22, would this be the code to run to generate it myself?

https://github.com/awohns/unified_genealogy_paper/blob/master/all-data/Makefile

And how much RAM would you guesstimate I'd need to run it successfully?

Thanks.

[hyanwong]

We'll definitely be releasing a dated TS in the next month or so. Otherwise you can try generating it. That is indeed the correct Makefile. Chromosome 22 is quite small, so you should be able to do that without a souped-up cluster, but 1GB might be pushing it a bit. You could always try a smaller portion of a larger chromosome to test it out (the function sample_data.subset() is useful for that). Note that there has been a (minor) bug fix to tsinfer recently, so I would advise trying this with the GitHub version of tsinfer installed (pip3 install git+https://github.com/tskit-dev/tsinfer)

[clawsoon]

The visualization code has been keeping me busy enough as it is, so using your release when it comes out will be perfect for me. Thanks!