Skip to content
Snippets Groups Projects
Commit 2df9d804 authored by Remi Hellequin's avatar Remi Hellequin
Browse files

Update README. Change default q in PBS scripts. Change temp files name in gitignore.

parent e6587f3b
Branches master
No related tags found
No related merge requests found
transformers
debug_squad
runs
debug_squad_*
cached_*
squad_train_*
......@@ -14,6 +14,7 @@ fusion-output -f <jobid> # watch the job logs during
Fusion supercomputer documentation : https://mesocentre.pages.centralesupelec.fr/user_doc/
Transformers on github : https://github.com/huggingface/transformers
The documentation for `run_squad.py` can be found here : https://huggingface.co/transformers/examples.html#squad
## Configure environment
......@@ -39,13 +40,17 @@ Run the network training
```bash
qsub <pbs_script>.pbs
```
> Some temporary data is written in directory `--output_dir` (`./debug_squad/`). You may have to clean the directory manually before relaunching the training `rm -r ./debug_squad/`
Two training examples :
Two training examples are provided :
- `single_gpu_training.pbs` : train the network on a single GPUs
- `dual_gpu_training.pbs` : train the network on a two GPUs
Notes :
- Some temporary data is written in directory `--output_dir` (`./debug_squad/`). You may have to clean the directory manually before relaunching the training `rm -r ./debug_squad/`
- During the TP sessions, you can use the reservation `isiaq` instead of the `gpuq` by commenting/decommenting lines beginning with `#PBS -q`)
## Misc notes
### Squad dataset location
......
......@@ -4,7 +4,8 @@
#PBS -l walltime=02:00:00
#PBS -l select=1:ncpus=24:ngpus=2:mem=20gb
#PBS -q gpuq
#PBS -P test
##PBS -q isiaq
#PBS -P isia
# Go to the current directory
cd $PBS_O_WORKDIR
......
......@@ -4,7 +4,8 @@
#PBS -l walltime=02:00:00
#PBS -l select=1:ncpus=12:ngpus=1:mem=20gb
#PBS -q gpuq
#PBS -P test
##PBS -q isiaq
#PBS -P isia
# Go to the current directory
cd $PBS_O_WORKDIR
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment