GSoC Week 3

03 Jun 2018

Reading time ~2 minutes

Google Summer of Coding Week 3

Hi, all. 😀

The third week has now ended. Let’s talk about some important points.

I couldn’t code for two-three consecutive days this week as I had an exam. But still, I was able to do what I had planned, to some extent.

PaddlePaddle Docker appliance on the CWRU HPC

This week, I gave a try for Docker appliance following the steps Prof. Peter did before. I had an idea that the image itself has no problem. And what we need to do is to modify $HOME path inside the image. It turned out that my idea is right. As a result, we can finally run DeepSpeech2 on CWRU HPC! Here’s what I’ve done:

1.Login

CynthiaHsus-MacBook-Pro:~ suwi$ ssh sxx186@redhen1.case.edu
Last login: Mon May 28 05:12:11 2018 from 202.120.19.74
sxx186@redhen1:~$ ssh sxx186@rider.case.edu
Last login: Mon May 28 05:12:13 2018 from redhen1.cosi.cwru.edu
[sxx186@hpclogin ~]$ srun -p gpu -C gpuk40 --gres=gpu:1 --pty bash
srun: job 7813756 queued and waiting for resources
srun: job 7813756 has been allocated resources
[sxx186@gpu028t ~]$ module load singularity/2.5.1

2.Download the Docker image and rename it as you like

[sxx186@gpu028t ~]$ singularity pull docker://paddlepaddle/deep_speech:latest-gpu
WARNING: pull for Docker Hub is not guaranteed to produce the
WARNING: same image on repeated pull. Use Singularity Registry
WARNING: (shub://) to pull exactly equivalent images.
Docker image path: index.docker.io/paddlepaddle/deep_speech:latest-gpu
Cache folder set to /home/sxx186/.singularity/docker
[17/17] |===================================| 100.0% 
Importing: base Singularity environment
Exploding layer: sha256:d5c6f90da05dc7e77d2e5fef63c341ab05ba2a03396ab5ae8f18814a7bbf5265.tar.gz
Exploding layer: sha256:bbbe761fcb565a007b458e09e08ecb88947f647f57be819a492a6b23694cefd8.tar.gz
Exploding layer: sha256:7afa5ede606fb1845e42f0d4816d2a7593a2b666ff9ca4722dcd2cff8a541acf.tar.gz
Exploding layer: sha256:f6b7253b56f434d6e4d97d259cb1007481cf670df2e04229e83cf37db33d96eb.tar.gz
Exploding layer: sha256:2b8db33536d447786cf57cca870fdd7d50d55fd67060c06252edc1c8db456a8f.tar.gz
Exploding layer: sha256:e0f506dd4a700abd7006016a537351f3e6e429a346bd8b457d91b80894f13884.tar.gz
Exploding layer: sha256:21ae5b69f65daa289d53eca8f567ec983773aa10c79290326de8e525d1721ec2.tar.gz
Exploding layer: sha256:87bc59b3df1f85363c35d004ffdda7b4d9c9f0138159ec4a7fc11336c609c255.tar.gz
Exploding layer: sha256:8c5da5017d0a1fb008aa5b1b4ff7b8d4579ad01963c08855e1e6cb44274275e4.tar.gz
Exploding layer: sha256:624201d912d1758f0b40bfe94507b2c58032a1032034cdb3a2d6ab7b411bcf11.tar.gz
Exploding layer: sha256:65f018df44bdeee1c0b9cf20877862a6729b325bf3efd0d1dc8ba9c68b621239.tar.gz
Exploding layer: sha256:9c7a53ae7ba0b8e176b44dc8b8ac8ee2867bd6703a0e7c7300ab325f0d955a8a.tar.gz
Exploding layer: sha256:8eb318e8051e88cca6a629972690ad2b1e0a43b8673d51ec8ea0b45b28f16293.tar.gz
Exploding layer: sha256:9b014bae4feef5c6253d4b2cf0d76817649a605dbbb23a6c36e143aa5a537dda.tar.gz
Exploding layer: sha256:132470f5050df9b5570e5e6c24906a9d23b44f9bfb57faf02c9f3887f34bd5b9.tar.gz
Exploding layer: sha256:fd77b9e6c0999686b6b57f51f72240669dcff93728ff2de7c17f448ee650a301.tar.gz
Exploding layer: sha256:1257e2a32bf9be4faafba2b0275486f51c3878bab8d0fb9d3099677f39cb4520.tar.gz
Exploding layer: sha256:11f2a6377069ab1f79d93ca05a42f15e0569d3582e6383f6aabb83efac99187b.tar.gz
WARNING: Building container as an unprivileged user. If you run this container as root
WARNING: it may be missing some functionality.
Building Singularity image...
Singularity container built: ./deep_speech-latest-gpu.simg
Cleaning up...
Done. Container is at: ./deep_speech-latest-gpu.simg
[sxx186@gpu028t ~]$ git clone https://github.com/PaddlePaddle/DeepSpeech.git
Cloning into 'DeepSpeech'...
remote: Counting objects: 2420, done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 2420 (delta 4), reused 6 (delta 3), pack-reused 2408
Receiving objects: 100% (2420/2420), 888.18 KiB | 0 bytes/s, done.
Resolving deltas: 100% (1563/1563), done.

mv deep_speech-suwi.simg DeepSpeech/

3.Get into the image and set $HOME

cd /mnt/rds/redhen/gallina/Singularity/DeepSpeech2/DeepSpeech/
singularity shell -e -H /mnt/rds/redhen/gallina/Singularity/DeepSpeech2/DeepSpeech/ deep_speech-suwi.simg
Singularity: Invoking an interactive shell within container...

unset HOME
export HOME="/mnt/rds/redhen/gallina/Singularity/DeepSpeech2/DeepSpeech/"

4.Run the code(run_data.sh as an example, and you can see Tiny data preparation done.)

cd examples/aishell/
sh run_data.sh

Conclusion

And thus, the third week ends. 12 – 3 = 9 weeks of awesomeness still left. 😉I will be back with 4th week’s blog soon. Thank you for reading. Bye.

GSoC