Hello, please help me solve this problem, thank you!!!
I’m using a slurm cluster with 400g of memory set up, but GTDB only seems to take up virtual memory
[2024-06-04 15:27:04] INFO: GTDB-Tk v2.3.2
[2024-06-04 15:27:04] INFO: gtdbtk classify_wf --genome_dir /public/home/zhangmy/10.GDTB_output/gtdb_input/ --out_dir /public/home/zhangmy/10.GDTB_output/gtdb_output/ --extension fa --skip_ani_screen --prefix tax --cpus 50
[2024-06-04 15:27:04] INFO: Using GTDB-Tk reference data version r214: /public/home/zhangmy/database/GTDB_tk/release214
[2024-06-04 15:27:04] INFO: Identifying markers in 8 genomes with 50 threads.
[2024-06-04 15:27:04] TASK: Running Prodigal V2.6.3 to identify genes.
[2024-06-04 15:27:30] INFO: Completed 8 genomes in 25.18 seconds (3.15 seconds/genome).
[2024-06-04 15:27:30] TASK: Identifying TIGRFAM protein families.
[2024-06-04 15:27:36] INFO: Completed 8 genomes in 6.39 seconds (1.25 genomes/second).
[2024-06-04 15:27:36] TASK: Identifying Pfam protein families.
[2024-06-04 15:27:37] INFO: Completed 8 genomes in 0.54 seconds (14.76 genomes/second).
[2024-06-04 15:27:37] INFO: Annotations done using HMMER 3.4 (Aug 2023).
[2024-06-04 15:27:37] TASK: Summarising identified marker genes.
[2024-06-04 15:27:37] INFO: Completed 8 genomes in 0.23 seconds (34.72 genomes/second).
[2024-06-04 15:27:37] INFO: Done.
[2024-06-04 15:27:40] INFO: Aligning markers in 8 genomes with 50 CPUs.
[2024-06-04 15:27:40] INFO: Processing 8 genomes identified as bacterial.
[2024-06-04 15:27:52] INFO: Read concatenated alignment for 80,789 GTDB genomes.
[2024-06-04 15:27:52] TASK: Generating concatenated alignment for each marker.
[2024-06-04 15:27:56] ERROR: Uncontrolled exit resulting from an unexpected error.
================================================================================
EXCEPTION: OSError
MESSAGE: [Errno 12] Cannot allocate memory
Traceback (most recent call last):
File “/public/home/zhangmy/conda/envs/gtdbtk-2.3.2/lib/python3.8/site-packages/gtdbtk/main.py”, line 102, in main
gt_parser.parse_options(args)
File “/public/home/zhangmy/conda/envs/gtdbtk-2.3.2/lib/python3.8/site-packages/gtdbtk/main.py”, line 1180, in parse_options
self.align(options)
File “/public/home/zhangmy/conda/envs/gtdbtk-2.3.2/lib/python3.8/site-packages/gtdbtk/main.py”, line 367, in align
reports = markers.align(options.identify_dir,
File “/public/home/zhangmy/conda/envs/gtdbtk-2.3.2/lib/python3.8/site-packages/gtdbtk/markers.py”, line 582, in align
user_msa = align.align_marker_set(
File “/public/home/zhangmy/conda/envs/gtdbtk-2.3.2/lib/python3.8/site-packages/gtdbtk/pipeline/align.py”, line 219, in align_marker_set
single_copy_hits = get_single_copy_hits(gid_dict, copy_number_file, cpus)
File “/public/home/zhangmy/conda/envs/gtdbtk-2.3.2/lib/python3.8/site-packages/gtdbtk/pipeline/align.py”, line 78, in get_single_copy_hits
with mp.Pool(processes=cpus) as pool:
File “/public/home/zhangmy/conda/envs/gtdbtk-2.3.2/lib/python3.8/multiprocessing/context.py”, line 119, in Pool
return Pool(processes, initializer, initargs, maxtasksperchild,
File “/public/home/zhangmy/conda/envs/gtdbtk-2.3.2/lib/python3.8/multiprocessing/pool.py”, line 212, in init
self._repopulate_pool()
File “/public/home/zhangmy/conda/envs/gtdbtk-2.3.2/lib/python3.8/multiprocessing/pool.py”, line 303, in _repopulate_pool
return self._repopulate_pool_static(self._ctx, self.Process,
File “/public/home/zhangmy/conda/envs/gtdbtk-2.3.2/lib/python3.8/multiprocessing/pool.py”, line 326, in _repopulate_pool_static
w.start()
File “/public/home/zhangmy/conda/envs/gtdbtk-2.3.2/lib/python3.8/multiprocessing/process.py”, line 121, in start
self._popen = self._Popen(self)
File “/public/home/zhangmy/conda/envs/gtdbtk-2.3.2/lib/python3.8/multiprocessing/context.py”, line 277, in _Popen
return Popen(process_obj)
File “/public/home/zhangmy/conda/envs/gtdbtk-2.3.2/lib/python3.8/multiprocessing/popen_fork.py”, line 19, in init
self._launch(process_obj)
File “/public/home/zhangmy/conda/envs/gtdbtk-2.3.2/lib/python3.8/multiprocessing/popen_fork.py”, line 70, in _launch
self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory
Hello,
Could you please provide information about the slurm script you are running to submit your jobs?
Do you have any errors in the slurm output files ( .out and/or .err )?
Thanks,
Pierre