Transferring files from the Epigenetics Core servers to blastula

• On blastula, open a terminal and SSH into the Epigenetic's core's server using UND's <firstname>.<lastname> login and UND password.

blastula {46} ssh <firsname>.<lastname>@epicenter.med.und.edu                  
<firstname>.<lastname>@epicenter.med.und.edu's password:

• Change directory to /data2/manu for our lab's data directory.

[manu.manu@epicenter manu]$ cd /data2/manu

• Use ls -ltr and cd to navigate to the location of the data to be downloaded.

                      
[manu.manu@epicenter manu]$ ls -ltr
total 224
.
.
.
drwxrwsr-x  2 manu.manu           undmed_epigenetics_data_manu_group    249 May 23  2023 G2023_22
drwxrwsr-x  4 manu.manu           undmed_epigenetics_data_manu_group   4096 Jun  8  2023 G2022_47
drwxrwsr-x  3 manu.manu           undmed_epigenetics_data_manu_group     49 Nov  9  2023 GeoMX November 2023
drwxrwsr-x  9 manu.manu           undmed_epigenetics_data_manu_group   4096 Jun 11  2024 G2022_47_new
drwxrwsr-x  4 manu.manu           undmed_epigenetics_data_manu_group     60 Nov 18 10:16 G2024_79_96
drwxrwsr-x  4 manu.manu           undmed_epigenetics_data_manu_group     93 Feb 10 23:03 sunilNooti

[manu.manu@epicenter manu]$ cd sunilNooti/
[manu.manu@epicenter sunilNooti]$ ls -ltr
total 8
drwxrwsr-x 2 manu.manu undmed_epigenetics_data_manu_group 4096 Nov 17  2023 G2023_84
drwxrwsr-x 2 manu.manu undmed_epigenetics_data_manu_group 4096 Feb 10 23:03 G2025_3
[manu.manu@epicenter sunilNooti]$ cd G2025_3/
[manu.manu@epicenter G2025_3]$ ls -ltr
total 0
lrwxrwxrwx 1 manu.manu undmed_epigenetics_data_manu_group 82 Feb 10 23:03 G2025_3_4_S4_L002_R2_001.fastq.gz -> /data2/core/raw_data_sync_talon/manumanu/G2025_3/G2025_3_4_S4_L002_R2_001.fastq.gz
lrwxrwxrwx 1 manu.manu undmed_epigenetics_data_manu_group 82 Feb 10 23:03 G2025_3_4_S4_L002_R1_001.fastq.gz -> /data2/core/raw_data_sync_talon/manumanu/G2025_3/G2025_3_4_S4_L002_R1_001.fastq.gz
lrwxrwxrwx 1 manu.manu undmed_epigenetics_data_manu_group 82 Feb 10 23:03 G2025_3_3_S3_L002_R2_001.fastq.gz -> /data2/core/raw_data_sync_talon/manumanu/G2025_3/G2025_3_3_S3_L002_R2_001.fastq.gz
.
.
.

• This confirms that the data files are contained in the G2025_3 directory. Use pwd to print the full path of the directory containing the data.

                      
[manu.manu@epicenter G2025_3]$ pwd
/data2/manu/sunilNooti/G2025_3

• Log out of epicenter and cd to the directory in /labcommon/SequenceData where the data would be stored. Use mkdir to create directories as needed.

                      
[manu.manu@epicenter G2025_3]$ logout
Connection to epicenter.med.und.edu closed.
(base) blastula {2} cd /labcommon/SequenceData/
(base) blastula {3} ls
Amp-seq  ATAC-Seq  CUTNRUN  DNA-seq  Genome  HiC  HiDRA_MiSeq  Index  MAPit  Public  rgtdata  RNAseq  scRNA-Seq  temp_GEOSubmission
(base) blastula {16} cd RNAseq
(base) blastula {61} ls
PUER_timecourse_2017

(base) blastula {18} mkdir MPRA_TrialRun_2025
(base) blastula {19} cd MPRA_TrialRun_2025

• Use rsync to copy over the data from epicenter. First run it in the "dry run" mode to make sure everything looks OK and that it is not deleting or overwriting anything.

                      
(base) blastula {38} rsync -rLptDvz --rsh=ssh --dry-run manu.manu@epicenter.med.und.edu:/data2/manu/sunilNooti/G2025_3 .
manu.manu@epicenter.med.und.edu's password: 
receiving incremental file list
G2025_3/
G2025_3/G2025_3_1_S1_L002_R1_001.fastq.gz
G2025_3/G2025_3_1_S1_L002_R2_001.fastq.gz
G2025_3/G2025_3_2_S2_L002_R1_001.fastq.gz
G2025_3/G2025_3_2_S2_L002_R2_001.fastq.gz
G2025_3/G2025_3_3_S3_L002_R1_001.fastq.gz
G2025_3/G2025_3_3_S3_L002_R2_001.fastq.gz
G2025_3/G2025_3_4_S4_L002_R1_001.fastq.gz
G2025_3/G2025_3_4_S4_L002_R2_001.fastq.gz
G2025_3/G2025_3_6_S5_L002_R1_001.fastq.gz
G2025_3/G2025_3_6_S5_L002_R2_001.fastq.gz
G2025_3/G2025_3_7_S6_L002_R1_001.fastq.gz
G2025_3/G2025_3_7_S6_L002_R2_001.fastq.gz
G2025_3/G2025_3_8_S7_L002_R1_001.fastq.gz
G2025_3/G2025_3_8_S7_L002_R2_001.fastq.gz
G2025_3/G2025_3_9_S8_L002_R1_001.fastq.gz
G2025_3/G2025_3_9_S8_L002_R2_001.fastq.gz
G2025_3/md5allsum.txt
G2025_3/post_CHECKSUMS.txt
G2025_3/post_CHECKSUMS1.txt

sent 85 bytes  received 648 bytes  97.73 bytes/sec
total size is 2,270,443,621  speedup is 3,097,467.42 (DRY RUN)

• Now run rsync for real without the --dry-run option.

   
(base) blastula {39} rsync -rLptDvz --rsh=ssh manu.manu@epicenter.med.und.edu:/data2/manu/sunilNooti/G2025_3 .
manu.manu@epicenter.med.und.edu's password: 
receiving incremental file list
G2025_3/
G2025_3/G2025_3_1_S1_L002_R1_001.fastq.gz
G2025_3/G2025_3_1_S1_L002_R2_001.fastq.gz
G2025_3/G2025_3_2_S2_L002_R1_001.fastq.gz
G2025_3/G2025_3_2_S2_L002_R2_001.fastq.gz
G2025_3/G2025_3_3_S3_L002_R1_001.fastq.gz
G2025_3/G2025_3_3_S3_L002_R2_001.fastq.gz
G2025_3/G2025_3_4_S4_L002_R1_001.fastq.gz
G2025_3/G2025_3_4_S4_L002_R2_001.fastq.gz
G2025_3/G2025_3_6_S5_L002_R1_001.fastq.gz
G2025_3/G2025_3_6_S5_L002_R2_001.fastq.gz
G2025_3/G2025_3_7_S6_L002_R1_001.fastq.gz
G2025_3/G2025_3_7_S6_L002_R2_001.fastq.gz
G2025_3/G2025_3_8_S7_L002_R1_001.fastq.gz
G2025_3/G2025_3_8_S7_L002_R2_001.fastq.gz
G2025_3/G2025_3_9_S8_L002_R1_001.fastq.gz
G2025_3/G2025_3_9_S8_L002_R2_001.fastq.gz
G2025_3/md5allsum.txt
G2025_3/post_CHECKSUMS.txt
G2025_3/post_CHECKSUMS1.txt

sent 389 bytes  received 2,256,119,450 bytes  20,234,258.65 bytes/sec
total size is 2,270,443,621  speedup is 1.01

• Check the MD5 sums to make sure the copied data do not have errors.

  
(base) blastula {50} ls
G2025_3
(base) blastula {51} cd G2025_3/
(base) blastula {52} md5sum -c md5allsum.txt 
G2025_3_1_S1_L002_R1_001.fastq.gz: OK
G2025_3_1_S1_L002_R2_001.fastq.gz: OK
G2025_3_2_S2_L002_R1_001.fastq.gz: OK
G2025_3_2_S2_L002_R2_001.fastq.gz: OK
G2025_3_3_S3_L002_R1_001.fastq.gz: OK
G2025_3_3_S3_L002_R2_001.fastq.gz: OK
G2025_3_4_S4_L002_R1_001.fastq.gz: OK
G2025_3_4_S4_L002_R2_001.fastq.gz: OK
G2025_3_6_S5_L002_R1_001.fastq.gz: OK
G2025_3_6_S5_L002_R2_001.fastq.gz: OK
G2025_3_7_S6_L002_R1_001.fastq.gz: OK
G2025_3_7_S6_L002_R2_001.fastq.gz: OK
G2025_3_8_S7_L002_R1_001.fastq.gz: OK
G2025_3_8_S7_L002_R2_001.fastq.gz: OK
G2025_3_9_S8_L002_R1_001.fastq.gz: OK
G2025_3_9_S8_L002_R2_001.fastq.gz: OK
(base) blastula {53}