DATA LEVEL

DATA CONTACT PI

Genome Sequence

Carlos Araya (claraya@gmail.com) ; Philip Cayting (pcayting@stanford.edu)

http://encodedcc.sdsc.edu/ftp/modENCODE_VS_ENCODE/Regulation/Worm/alignments/distinctTagAlign/reps/

Carlos Araya (claraya@gmail.com) ; Anshul Kundaje (anshul@kundaje.net)

https://docs.google.com/spreadsheet/ccc?key=0Algk3BSZDYzgdDlYNU00d2p3azJyZWlrZ09OQXNXTGc#gid=0

Carlos Araya (claraya@gmail.com) ; Philip Cayting (pcayting@stanford.edu)

http://encodedcc.sdsc.edu/ftp/modENCODE_VS_ENCODE/Regulation/Worm/mappability/globalmap_k20tok54.tgz

A position 'i' on a particular genomic strand 's' is considered uniquely mappable for a read-length 'k' if the k-mer starting at 'i' on strand 's' maps uniquely i.e. only to position 'i' on strand 's' (no mismatches allowed). There are other ways to define mappability e.g. allowing for mismatches but this is basically an "optimistic" idealized mappability mask not accounting for mismatches.

A whole genome index is created and the Bowtie mapper was used to try to map each k-mer against both strands of the genome.

globalmap_k20tok54.tgz file contains binary files representing uniqueness maps for each chromosome for all read-lengths ranging from 20 to 54 (encoded in a single file for each chromosome)
(a) The files are in uint8 (unsigned 8 bit integers) binary formats (saves disk space)
(b) Each file is basically a vector of unsigned 8bit integers that is the length of the chromosome. The elements of the vector are >= 0 (taking values 0 or 20 to 54)
(c) A value of 'x' at a position means that position is PERFECTLY unique in the genome for all k-mers of length >= x starting at that position on the + strand
(d) A value of 0 at a position means that position is not unique for any of the k-mer lengths (k=20 to 54)
(e) In order to obtain the uniqueness map for a particular read-length 'k', simply perform the following operation on each element of the vector (vector > 0) & (vector <= k)
(f) In order to obtain the uniquness map for the - strand, you simply need to right-shift the vector by <k-1>.
i.e. if position 1 is UNIQUE on the + strand for read-length <k=3> then it implies position 3 is UNIQUE on the - strand

===============================================
How to read the files in a programming language such as matlab/octave
===============================================
%First gunzip and untar the globalmap_k20tok54.tgz file
%You will see one file for each chromosome e.g. chr1.uint8.unique
% Read the files as a contiguous binary vector of unsigned 8 bit integers

tmp_uMap = fopen('chr1.uint8.unique','r');
uMapdata = fread(tmp_uMap,'*uint8');
fclose(tmp_uMap);

% You can similarly read the files in any other programming language as a vector of unsigned 8bit integers. Convert to doubles if you like (although this is a waste of memory) or write it out as a text file if you prefer

Signal tracks are generated for each dataset using MACSv2's signal processing module. Signal tracks represent ChIP signal compared to input control signal.
- FoldEnrichment: The first type of track represents the fold enrichment of ChIP over input. This type of track is useful for detecting and analyzing regions with moderate to low enrichment.
- Does NOT correct for local mappability
- Does NOT differentiate between "missing data" at unmappable locations and true 0 signal.
- The signal is not smoothed. Reads are extended to predominant fragment length.
- It is recommended to smooth the signal before using it unless you are averaging over multiple sites (.e.g aggregation plots)
- Separate tracks are generated for individual replicates and pooled data.

Chao Cheng (Chao Cheng <chengchao12@gmail.com>)

Carlos Araya (claraya@gmail.com) ; Alan Boyle (aboyle@stanford.edu)

Genome Sequence

https://docs.google.com/spreadsheet/ccc?key=0Algk3BSZDYzgdDU3cXVVMHdQeHRTUWtnYk1aSG13NEE&pli=1#gid=4

http://www.broadinstitute.org/~anshul/projects/encode/rawdata/umap/dm3_build5/dm3_build5.all.globalmap_k20tok54.tgz

A position 'i' on a particular genomic strand 's' is considered uniquely mappable for a read-length 'k' if the k-mer starting at 'i' on strand 's' maps uniquely i.e. only to position 'i' on strand 's' (no mismatches allowed). There are other ways to define mappability e.g. allowing for mismatches but this is basically an "optimistic" idealized mappability mask not accounting for mismatches.

A whole genome index (except for the human female mask for which chrY was excluded from the index) is created and the Bowtie mapper was used to try to map each k-mer against both strands of the genome.

Each <organism>.<sex>.globalmap_k20tok54.tgz file contains binary files representing uniqueness maps for each chromosome for all read-lengths ranging from 20 to 54 (encoded in a single file for each chromosome)
(a) The files are in uint8 (unsigned 8 bit integers) binary formats (saves disk space)
(b) Each file is basically a vector of unsigned 8bit integers that is the length of the chromosome. The elements of the vector are >= 0 (taking values 0 or 20 to 54)
(c) A value of 'x' at a position means that position is PERFECTLY unique in the genome for all k-mers of length >= x starting at that position on the + strand
(d) A value of 0 at a position means that position is not unique for any of the k-mer lengths (k=20 to 54)
(e) In order to obtain the uniqueness map for a particular read-length 'k', simply perform the following operation on each element of the vector (vector > 0) & (vector <= k)
(f) In order to obtain the uniquness map for the - strand, you simply need to right-shift the vector by <k-1>.
i.e. if position 1 is UNIQUE on the + strand for read-length <k=3> then it implies position 3 is UNIQUE on the - strand

===============================================
How to read the files in a programming language such as matlab/octave
===============================================
%First gunzip and untar the globalmap_k20tok54.tgz file
%You will see one file for each chromosome e.g. chr1.uint8.unique
% Read the files as a contiguous binary vector of unsigned 8 bit integers

tmp_uMap = fopen('chr1.uint8.unique','r');
uMapdata = fread(tmp_uMap,'*uint8');
fclose(tmp_uMap);

% You can similarly read the files in any other programming language as a vector of unsigned 8bit integers. Convert to doubles if you like (although this is a waste of memory) or write it out as a text file if you prefer

Signal tracks are generated for each dataset using MACSv2's signal processing module. Signal tracks represent ChIP signal compared to input control signal.
- FoldEnrichment: The first type of track represents the fold enrichment of ChIP over input. This type of track is useful for detecting and analyzing regions with moderate to low enrichment.
- Does NOT correct for local mappability
- Does NOT differentiate between "missing data" at unmappable locations and true 0 signal.
- The signal is not smoothed. Reads are extended to predominant fragment length.
- It is recommended to smooth the signal before using it unless you are averaging over multiple sites (.e.g aggregation plots)
- Separate tracks are generated for individual replicates and pooled data.

Chao Cheng (Chao Cheng <chengchao12@gmail.com>)

Carlos Araya (claraya@gmail.com) ; Alan Boyle (aboyle@stanford.edu)

Genome Sequence

FASTQ and BAM files can be downloaded from the URL. Different labs used different mappers and mapping strategies. Hence, these files should be filtered to standardize them.

https://docs.google.com/spreadsheet/ccc?key=0Am6FxqAtrFDwdHdRcHNQUy03SjBoSVMxdUNyZV9Rdnc#gid=9

http://hgdownload-test.cse.ucsc.edu/goldenPath/hg19/encodeDCC/wgEncodeMapability/wgEncodeDacMapabilityConsensusExcludable.bed.gz

http://www.broadinstitute.org/~anshul/projects/encode/rawdata/peaks_spp/mar2012/distinct/idrOptimalBlackListFilt/

http://www.broadinstitute.org/~anshul/projects/encode/rawdata/peaks_spp/mar2012/distinct/combrep/regionPeak/

http://www.broadinstitute.org/~anshul/projects/encode/rawdata/signal/mar2012/pooledReps/bigwig/macs2signal/foldChange/

Signal tracks are generated for each dataset using MACSv2's signal processing module. Signal tracks represent ChIP signal compared to input control signal.
- FoldEnrichment: The first type of track represents the fold enrichment of ChIP over input. This type of track is useful for detecting and analyzing regions with moderate to low enrichment.
- Does NOT correct for local mappability
- Does NOT differentiate between "missing data" at unmappable locations and true 0 signal.
- The signal is not smoothed. Reads are extended to predominant fragment length.
- It is recommended to smooth the signal before using it unless you are averaging over multiple sites (.e.g aggregation plots)
- Separate tracks are generated for individual replicates and pooled data.

Human (hg19)
Male genome (with chrY)

A position 'i' on a particular genomic strand 's' is considered uniquely mappable for a read-length 'k' if the k-mer starting at 'i' on strand 's' maps uniquely i.e. only to position 'i' on strand 's' (no mismatches allowed). There are other ways to define mappability e.g. allowing for mismatches but this is basically an "optimistic" idealized mappability mask not accounting for mismatches.

A whole genome index (except for the human female mask for which chrY was excluded from the index) is created and the Bowtie mapper was used to try to map each k-mer against both strands of the genome.

Each <organism>.<sex>.globalmap_k20tok54.tgz file contains binary files representing uniqueness maps for each chromosome for all read-lengths ranging from 20 to 54 (encoded in a single file for each chromosome)
(a) The files are in uint8 (unsigned 8 bit integers) binary formats (saves disk space)
(b) Each file is basically a vector of unsigned 8bit integers that is the length of the chromosome. The elements of the vector are >= 0 (taking values 0 or 20 to 54)
(c) A value of 'x' at a position means that position is PERFECTLY unique in the genome for all k-mers of length >= x starting at that position on the + strand
(d) A value of 0 at a position means that position is not unique for any of the k-mer lengths (k=20 to 54)
(e) In order to obtain the uniqueness map for a particular read-length 'k', simply perform the following operation on each element of the vector (vector > 0) & (vector <= k)
(f) In order to obtain the uniquness map for the - strand, you simply need to right-shift the vector by <k-1>.
i.e. if position 1 is UNIQUE on the + strand for read-length <k=3> then it implies position 3 is UNIQUE on the - strand

===============================================
How to read the files in a programming language such as matlab/octave
===============================================
%First gunzip and untar the globalmap_k20tok54.tgz file
%You will see one file for each chromosome e.g. chr1.uint8.unique
% Read the files as a contiguous binary vector of unsigned 8 bit integers

tmp_uMap = fopen('chr1.uint8.unique','r');
uMapdata = fread(tmp_uMap,'*uint8');
fclose(tmp_uMap);

% You can similarly read the files in any other programming language as a vector of unsigned 8bit integers. Convert to doubles if you like (although this is a waste of memory) or write it out as a text file if you prefer

Human (hg19)
Female genome (no chrY)

A position 'i' on a particular genomic strand 's' is considered uniquely mappable for a read-length 'k' if the k-mer starting at 'i' on strand 's' maps uniquely i.e. only to position 'i' on strand 's' (no mismatches allowed). There are other ways to define mappability e.g. allowing for mismatches but this is basically an "optimistic" idealized mappability mask not accounting for mismatches.

A whole genome index (except for the human female mask for which chrY was excluded from the index) is created and the Bowtie mapper was used to try to map each k-mer against both strands of the genome.

Each <organism>.<sex>.globalmap_k20tok54.tgz file contains binary files representing uniqueness maps for each chromosome for all read-lengths ranging from 20 to 54 (encoded in a single file for each chromosome)
(a) The files are in uint8 (unsigned 8 bit integers) binary formats (saves disk space)
(b) Each file is basically a vector of unsigned 8bit integers that is the length of the chromosome. The elements of the vector are >= 0 (taking values 0 or 20 to 54)
(c) A value of 'x' at a position means that position is PERFECTLY unique in the genome for all k-mers of length >= x starting at that position on the + strand
(d) A value of 0 at a position means that position is not unique for any of the k-mer lengths (k=20 to 54)
(e) In order to obtain the uniqueness map for a particular read-length 'k', simply perform the following operation on each element of the vector (vector > 0) & (vector <= k)
(f) In order to obtain the uniquness map for the - strand, you simply need to right-shift the vector by <k-1>.
i.e. if position 1 is UNIQUE on the + strand for read-length <k=3> then it implies position 3 is UNIQUE on the - strand

===============================================
How to read the files in a programming language such as matlab/octave
===============================================
%First gunzip and untar the globalmap_k20tok54.tgz file
%You will see one file for each chromosome e.g. chr1.uint8.unique
% Read the files as a contiguous binary vector of unsigned 8 bit integers

tmp_uMap = fopen('chr1.uint8.unique','r');
uMapdata = fread(tmp_uMap,'*uint8');
fclose(tmp_uMap);

% You can similarly read the files in any other programming language as a vector of unsigned 8bit integers. Convert to doubles if you like (although this is a waste of memory) or write it out as a text file if you prefer

Chao Cheng (Chao Cheng <chengchao12@gmail.com>)

Chao Cheng (Chao Cheng <chengchao12@gmail.com>)

Carlos Araya (claraya@gmail.com) ; Alan Boyle (aboyle@stanford.edu)

Genome Sequence

Philip Cayting (pcayting@stanford.edu); Alan Boyle ; Yong Cheng

https://docs.google.com/spreadsheet/ccc?key=0Ao3-Or4FCMJEdFpPY2lwWnlZTV92MUNLOHYxbEl4Vnc#gid=0

Philip Cayting (pcayting@stanford.edu); Alan Boyle ; Yong Cheng

Philip Cayting (pcayting@stanford.edu); Alan Boyle ; Yong Cheng

Philip Cayting (pcayting@stanford.edu); Alan Boyle ; Yong Cheng

A position 'i' on a particular genomic strand 's' is considered uniquely mappable for a read-length 'k' if the k-mer starting at 'i' on strand 's' maps uniquely i.e. only to position 'i' on strand 's' (no mismatches allowed). There are other ways to define mappability e.g. allowing for mismatches but this is basically an "optimistic" idealized mappability mask not accounting for mismatches.

A whole genome index (except for the human female mask for which chrY was excluded from the index) is created and the Bowtie mapper was used to try to map each k-mer against both strands of the genome.

Each <organism>.<sex>.globalmap_k20tok54.tgz file contains binary files representing uniqueness maps for each chromosome for all read-lengths ranging from 20 to 54 (encoded in a single file for each chromosome)
(a) The files are in uint8 (unsigned 8 bit integers) binary formats (saves disk space)
(b) Each file is basically a vector of unsigned 8bit integers that is the length of the chromosome. The elements of the vector are >= 0 (taking values 0 or 20 to 54)
(c) A value of 'x' at a position means that position is PERFECTLY unique in the genome for all k-mers of length >= x starting at that position on the + strand
(d) A value of 0 at a position means that position is not unique for any of the k-mer lengths (k=20 to 54)
(e) In order to obtain the uniqueness map for a particular read-length 'k', simply perform the following operation on each element of the vector (vector > 0) & (vector <= k)
(f) In order to obtain the uniquness map for the - strand, you simply need to right-shift the vector by <k-1>.
i.e. if position 1 is UNIQUE on the + strand for read-length <k=3> then it implies position 3 is UNIQUE on the - strand

===============================================
How to read the files in a programming language such as matlab/octave
===============================================
%First gunzip and untar the globalmap_k20tok54.tgz file
%You will see one file for each chromosome e.g. chr1.uint8.unique
% Read the files as a contiguous binary vector of unsigned 8 bit integers

tmp_uMap = fopen('chr1.uint8.unique','r');
uMapdata = fread(tmp_uMap,'*uint8');
fclose(tmp_uMap);

% You can similarly read the files in any other programming language as a vector of unsigned 8bit integers. Convert to doubles if you like (although this is a waste of memory) or write it out as a text file if you prefer