Gitlab Bootcamp Kochi 2020

class: center, middle

.logo[![](https://github.com/Academany/academany-mkt/raw/master/fabacademy2017/logo/fabacademy.png)]

Kochi Januari 2020

# Gitlab Bootcamp

*** Henk Buursen ***

[henk@waag.org](mailto:henk@waag.org)

.logo_tiny[![](https://github.com/Academany/academany-mkt/raw/master/fabacademy2017/logo/academany.png)]

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

<div class="my-footer"><span>
<a rel="license" href="http://creativecommons.org/licenses/by-nc/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by-nc/4.0/80x15.png" /></a></a>
Henk Buursen - Waag - Fablab Amsterdam	</span></div>

# Goal of this workshop

.image_right[
![](./images/thisisgit.png)
]

1. better understanding of git
2. better understanding of gitlab
3. guiding students
4. CLI: less is more
5. reduce graphic sizes

#	AND ...

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

![](./images/dontpanik.jpg)

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# 1. Get a better understanding of git

- prepair yourself
   	- [Video Fiore's recitation (2018)](http://archive.fabacademy.org/2018/lectures/fab-20180122.html)
   	- [Slides of the recitation (2018)](http://archive.fabacademy.org/archives/2018/recitations/version-control.html)

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# 2. Get a better understanding of gitlab

- prepair yourself
	- [Video Fiore's recitation (2019)](https://vimeopro.com/academany/fab-2019/video/314643628)
	- [Slides of the recitation (2019)](http://fabacademy.org/2019/recitations/version-control/index.html)

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# 2a. CI / CD

- Continuous Integration (CI)
     - Continuous Delivery (CD)
     - Continuous Deployment (CD)

***
    Continuous Integration works by pushing small code chunks to your
    application’s code base hosted in a Git repository, and, to every push, run
    a pipeline of scripts to build, test, and validate the code changes before
    merging them into the main branch.

Continuous Delivery and Deployment consist of a step further CI, deploying
    your application to production at every push to the default branch of the
    repository.
*** 
    
---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# 2a. CI / CD gitlab-ci.yml

*** 
GitLab CI/CD is configured by a file called .gitlab-ci.yml placed at ***
the repository’s root. The scripts set in this file are executed by the
GitLab Runner. 
***

[
![](images/add_file_template_11_10b.png)
]
---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# 2b. Gitlab pages
 
***
    GitLab Pages is a feature that allows you to publish static websites
    directly from a repository in GitLab.
***

- To publish a website with Pages, you can use any Static Site Generator
     (SSG), such as Mkdocs, Jekyll, Hugo, Middleman, Harp, Hexo, and Brunch,
     just to name a few. You can also publish any website written directly in
     plain HTML, CSS, and JavaScript.

- Pages does not support dynamic server-side processing, for instance, as
     .php and .asp requires.

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

#2b. Gitlab pages on fabcloud

- See what CI / CD is doing

![](images/cicd.png)

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

#2b. Gitlab pages on fabcloud

![](images/pages.png)

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

But!

</center>
</p>
***
will be rewitten by nginx to http://fabacademy.org/2018/labs/fablabamsterdam/students/henk-buursen
***
---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# 2a. gitlab.fabcloud.org

- issue tracker
    	- setup
	- global
	- class

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# 3. Guiding students 
	
gitlab fabcloud lessons prior years

- uncompressed images and videos!
	- building a hugh git history
		(commiting all files with minor changes again and again)
	- Git LFS (Large File Support)

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# 3a. Guiding students

- help them to understand and use CI / CD
    - Show Static Site Generator on fabcloud
    - Setup local repository and SSG
    - Explain workflows: 
	  - document local
	  - test local SSG
	  - push to fabcloud

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# 3a. ImageMagick and ffmpeg

- file sizes and image sizes
     - identify
     ![](./images/identify.png)
     - convert -resize
     ![](./images/convert.png)

- [http://academy.cba.mit.edu/classes/computer_design/image.html](http://academy.cba.mit.edu/classes/computer_design/image.html)
        - [http://academy.cba.mit.edu/classes/computer_design/video.html](http://academy.cba.mit.edu/classes/computer_design/video.html)

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# 3a. Building  hugh git repository

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# 3a. Filesizes and tools

- du -hs

![](images/du.png)

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# find
    
     - find . -type f -size +1M -exec ls -lh {} \;

![](images/find.png)
     
---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# bash

```bash

for i in G M K
do
  du -ah | grep [0-9]$i | sort -nr -k 1
done | head -n 20
````

![](images/for_i.png)

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# zsh

```bash
ls -ld -- **/*(.DOL[1])
````

![](images/zsh.png)

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# NCDU

- install ncdu, it’s command-line and gives you interactive folder listings

![](images/ncdu.png)

---

# Managing large files efficiently with Git LFS

Nothing special: some HTML, CSS, and JS files and a couple of small image
assets. However, until now, we haven't included our design assets (Photoshop,
Sketch, etc.). It makes a lot of sense to put your design assets under version
control, too.

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# However,

here's the catch: each time our student makes a change (no matter how small) to
this new Photoshop file, she will commit another 100 MB to the repository. Very
quickly, the repository will weigh tons of megabytes and soon gigabytes - which
makes cloning and managing it very tedious

---

# With LFS: Efficient large file handling

Of course, LFS cannot simply "magic away" all that large data: it accrues with every change and has to be saved. However, it shifts that burden to the remote server - allowing the local repository to stay relatively lean!

To make this possible, LFS uses a simple trick: it does not keep all of a file's versions in the local repository. Instead, it provides only the files that are necessary in the checked out revision, on demand.

But this poses an interesting question: if those huge files themselves are not present in your local repository… what is present instead? LFS saves lightweight pointers in place of real file data. When you check out a revision with such a pointer, LFS simply looks up the original file (possibly on the server if it's not in its own, special cache) and downloads it for you.

Thereby, you end up with only the files you really want - not a whole bunch of superfluous data that you might never need.

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# Video

Git Large File Storage - How to Work with Big Files

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# Installing LFS (on debian)

.image_right[
![](./images/lfs_debian.png)
]

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# Initialize LFS and track filetypes

![](./images/lfs_init.png)

![](./images/lfs_track.png)

![](./images/lfs_track_cat.png)

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# Commiting and pushing

![](./images/lfs_add_commit.png)

![](./images/lfs_push.png)

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# Result in gitlab

![](./images/lfs_gitlab_show.png)

---

# More on GIT LFS

- migrating existing repo to git LFS
	- file locking

[https://docs.gitlab.com/ee/administration/lfs/manage_large_binaries_with_git_lfs.html](https://docs.gitlab.com/ee/administration/lfs/manage_large_binaries_with_git_lfs.html)

---

# BFG Repo-Cleaner

The BFG is a simpler, faster alternative to git-filter-branch for cleansing bad data out of your Git repository history:

Removing Crazy Big Files
    Removing Passwords, Credentials & other Private data

---

# Usage

First clone a fresh copy of your repo, using the --mirror flag:

```bash
$ git clone --mirror git://example.com/some-big-repo.git
```
.footnote[.red.bold[*]This is a bare repo, which means your normal files won't
be visible, but it is a full copy of the Git database of your repository, and
at this point you should make a backup of it to ensure you don't lose anything.]

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# Clean

Now you can run the BFG to clean your repository up:

```bash
$ java -jar bfg.jar --strip-blobs-bigger-than 100M some-big-repo.git
```

.footnote[.red.bold[*]The BFG will update your commits and all branches and
tags so they are clean, but it doesn't physically delete the unwanted stuff.]

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# Examine

Examine the repo to make sure your history has been updated, and then use the
standard git gc command to strip out the unwanted dirty data, which Git will
now recognise as surplus to requirements:

```bash
$ cd some-big-repo.git
$ git reflog expire --expire=now --all && git gc --prune=now --aggressive
```

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>

# Push

Finally, once you're happy with the updated state of your repo, push it back up
(note that because your clone command used the --mirror flag, this push will
update all refs on your remote server):

```bash
$ git push
```

---
<div class="my-header">
<img src="images/fabacademy.png" style="height:30px;" />
</div>