slurm

Posted on Edited on

How do config R in Slurm

[TOC]

Conda environment

  • Create environment 
    1
    conda create -n R4.4-syf R=4.4.1 -y
    Create R environment using 4.4.1 version.
  • Check current environments 
    1
    conda env list
  • Activate & Deactivate environment
    1
    2
    3
    4
    5
    # activate
    conda activate env_name # change to env_name created before
    conda activate R4.4-syf # eg.
    # deactivate
    conda deactivate R4.4-syf
  • Delete env
    1
    conda remove -n env_name --all -y

Install jags

  1. Download and Tar
    Download package from here 
    1
    2
    3
    4
    5
    6
    7
    mkdir ./JAGS-4.3.2
    tar -zxf ./jags_4.3.2.orig.tar.gz
    cd ./JAGS-4.3.2
    ./configure
    make
    make check
    make install
  2. Conda install rjags
    After tar JAGS, conda install rjags in command.
    1
    conda install -c conda forge r-rjags
  3. Install package in R
    Activate R environment using command in shell: R
    1
    2
    3
    install.packages("R2jags")
    library(rjags)
    library(R2jags)

Running R script using Shell in Slurm

The test shell script: submit_test.sh

R script: test_hpc_parallel.R

To run the R script using conda environment, the shell script:

1
2
3
4
5
6
7
module purge
module load anaconda3/2021.11
conda activate /home/224030234/R/R4.4-syf

cd /home/224030234/R
#run the application:
Rscript test_hpc_parallel.R

Following error will occur:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run

    $ conda init <SHELL_NAME>

Currently supported shells are:
   - bash
   - fish
   - tcsh
   - xonsh
   - zsh
   - powershell

See 'conda init --help' for more information and options.

IMPORTANT: You may need to close and restart your shell after running 'conda init'.

To solve this, source conda.sh before activate, and use the full path for the R scipt.

The revised shell script:
1
2
3
4
5
6
7
module purge
module load anaconda3/2021.11
source ~/anaconda3/etc/profile.d/conda.sh
conda activate /home/224030234/R/R4.4-syf

#run the application:
Rscript /home/224030234/R/test_hpc_parallel.R

Others

  1. conda install freezing
    When install r-pkg, solving environment freezing often occurs, to solve this, replace conda install with mamba.

    What is mamba

  2. conda channels operations
    1
    2
    conda config --show channels # show current channels
    conda config --remove channels defaults # remove defaults channel
    To add new channels:
    1
    2
    3
    vim ./condarc
    # add in condarc via vim
    conda config --show channels # check channels added status

Reference

  1. infercnv安装全流程（包含jags等所有踩坑指南）
  2. conda常用命令
  3. shell脚本激活conda虚拟环境
  4. conda中如何移除默认源