# Build 22.04.5, WS 2024

This file:
* Build of `dsci-lab_22.04.5_ws24`, 2024-03-02
* Basis: Xubuntu 22.04.4 LTS (Jammy Jellyfish), 64 bit
* **admin user install**

previous versions:
* dsci-lab WS 2023: <http://www.jbusse.de/2023_ws_dsci-lab/>

Important: You are *not* requested to build the dsci-lab by yourself. Instead you are advised to download the fully configured dsci-lab as a ready-to-use virtual machine, see {doc}`t_download`.

However, if you are a lecturer or a deeply interested student you might want to learn how we set up the dsci-lab. Here are the steps.



## Install VM Virtual Box 7

* {ref}`install-virtualbox`

## Create the virtual XUbuntu machine

Download an ISO image:
* e. g. from <http://ftp.uni-kl.de/pub/linux/ubuntu-dvd/xubuntu/releases/22.04/release/>
* file `xubuntu-22.04.5-desktop-amd64.iso` | 2024-09-11  16:41 | 3.0G | Desktop image for 64-bit PC (AMD64) computers (standard download)
  * right click on file, "save as", save to disk (e.g. `/home/user/v/`)

Start Oracle VM Virtual Box Manager
 
* Maschine > Neu > Name: e.g. *dsci-lab_22.04.5_ws24*
* Ort: z.B. `/home/jb/VirtualBox VMs`
* Typ: Linux, Version: Ubuntu (64-bit)
* Experten-Modus
* Speichergröße: minimal 4096 MB, besser z.B. 16000 MB ;-) )
* Platte: Festplatte erzeugen > Erzeugen
* Dateityp der Festplatte: VDI
* Art der Speicherung: dynamisch alloziert
* Dateiname und Größe: hier mindestens 100 GB angeben > Erzeugen

Start machine *dsci-lab_22.04.5_ws24*. 
* Starten > Try or install XUbuntu

VirtualBox will start the ISO image:

* Select language, e.g. "Deutsch" >  "Install XUbuntu"
* Tastaturbelegung: German > German
* Aktualisierungen herunterladen
* Installationsart: Festplatte löschen und installieren (this will clean only your newly allocated virtual hard disc, NOT the disk of your host system;-) ); > Installieren
* Wo befinden Sie sich? > Berlin
* "Who are You?":
   * Name: Data Scientist
   * Name des Rechners: dsci-lab_22.04.5_ws24
   * Benutzername: dsci
   * password: dscidsci

A basic version of xubuntu will be installed.

"Die Installation ist abgeschlossen. Sie müssen jetzt den Rechner neu starten, um das System zu benutzen" > Jetzt neu starten
* Virtualbox will become black and "freeze". Close the Window: "X"-button > die virtuelle Maschine ausschalten
* Start the new machine again.
* Log in with user *dsci*, password *dscidsci*


<!--
Whisker-Menue > Einstellungen > Sprachen
* Unvollständige Sprachunterstützung: Sprachunterstützung nachinstallieren? > Ja; (on request enter password, i.e. "dscidsci")
* Sprache für Menues und Fenster (je nach Geschmack): das dsci-lab verwendet English ; "Ziehen Sie die Sprachen in die gewünschte Reihenfolge";  systemweit anwenden
-->


## Update Basic System

"A new Release 24.04.1 LTS is available, upgrade?" > **NO, never upgrade**, we want to stick with 22.04

"Aktualisierungsverwaltung" will start automatically and ask you to update the system: "Aktualisierte Anwendungen wurden seit der Veröffentlichung von Ubuntu 22.04 herausgegeben" > "jetzt installieren". 

Alternatively you may want to start Aktualisierungsverwaltung manually:
* [Whisker-Menu](https://wiki.ubuntuusers.de/Whisker_Menu/) > Einstellungen > Aktualisierungsverwaltung

To update your installation, regularly start "Aktualisierungsverwaltung". As an alternative to using "Aktualisierungsverwaltung" you can keep the system current with `apt`:

```sh
sudo apt update; sudo apt upgrade
```

Try it! Your system already should be up to date.

Restart the machine afterwards. Login again.

* "Update standard folders to current language?" > im dsci-lab wollen wir englische Ordnernamen, also "update names"

We will use the command line hereafter where possible. Get a new terminal by typing `Strg-Alt-t`. 



## VirtualBox Guest Extensions

To install new kernel modules - i.e. Oracle VirtualBox Guest Extensions - we need (the gnu C compiler) `gcc`:

```sh
sudo apt install gcc make perl htop
```

Oracle Virtual Box > Geräte > Gasterweiterungen einlegen: A window pops up, showing the directory `/media/data/VBox_GAs_6.1.42./` > rightclick on background, "Open Terminal here",  a new terminal opens. Type in:

```sh
sudo ./VBoxLinuxAdditions.run
```

Some new kernel modules will be built. To replace the running kernel modules with the new ones you have to reboot the VM:

2024-09-19: Machine says "VirtualBox Guest Additions: reloading kernel modules and services": The mouse cannot be used any more. However, CTRL-T works. We shut down the machine manually:



```sh
sudo reboot
```

Login again. You now should be able to resize the VM window. Try it!

Activate bidirectional Copy & Paste from Windows-Host to VB etc. 

```
Geräte > Gemeinsame Zwischenablage > bidirektional
```

After updating virtualbox on the host you might want to update the guest extensions on the guest. Simply repeat the steps above. Donn't forget to reboot your guest engine.



<!--
## Firefox


Xubuntu 22 comes with a Snap-Version of firefox. This causes some ugly configuration issues. We remove this (so called "rapid") version of firefox and echange it with a `.deb` Version:

* <https://www.omgubuntu.co.uk/2022/04/how-to-install-firefox-deb-apt-ubuntu-22-04>

Test Firefox: Whisker-Menu "Internetnavigator" launces Firefox. Navigate to <http://jbusse.de/dsci-lab/build.html>. Alternatively manually type in (and don't forget to add the ampersand "&" at the end of the line):

```sh
firefox http://jbusse.de/dsci-lab/build.html &
```
	
If your WLAN works properly, Firefox will open the page you are actually reading here in your new linux machine. This allows you to copy & paste the following commands to your terminal. Bookmark this page in Firefox by hitting CTRL-D.


## Firefox ESR

Install Firefox ESR: <https://ubuntuhandbook.org/index.php/2022/03/install-firefox-esr-ubuntu/>

```
sudo add-apt-repository ppa:mozillateam/ppa
sudo apt update
sudo apt install firefox-esr
```

-->

## Firefox Privacy

Start firefox. Hit the "ALT"-button to show the menu.

Edit > Settings (Bearbeiten > Einstellungen):
* Home > New Windows and Tabs (Startseite und neue Fenster): point default to `https://www.startpage.com/`
  * Inhalte des Firefox-Startbildschirms: privacy! ... but do not deactivate these widges (don't close your eyes), but go to
* Datenschutz und Sicherheit > Benuzerdefiniert > Cookies und Website-Daten
  * Cookies und Website-Daten beim Beenden von Firefox löschen: Check!
  * Hauptpasswort verwenden: Enter a save password!
  * Chronik: Firefox wird eine Chronik > nach benutzerdefinierten Einstellungen anlegen
    * "Die Chronik löschen, wenn Firefox geschlossen wird": Check!
  * Datenerhebung durch Firefox und deren Verwendung
    * (de-)select whatever you want

Extras > Add-Ons und Themes > Erweiterungen
  * Ghostery – datenschutzorientierter Werbeblocker
    * optionally configure Ghostery: "ausführen in privaten Fenstern erlauben"


<!-- View markdown with Firefox

(1) install Add-On: Extras > Add-Ons und Themes > Erweiterungen
  * Markdown Viewer Webext von Kulero, Cimbali

(2) add MIME type: <https://github.com/KeithLRobertson/markdown-viewer#support-for-local-files-on-linux>: 
Firefox on Linux may not know how to handle markdown files by default (see #2). There are a number of possible workarounds for this (see this SuperUser question for example). [...] Another workaround (which might cover other OSs as well), is to edit Firefox’s private mime types. These mime types are stored in a file indicated by helpers.private_mime_types_file, by default it is `~/.mime.types`:

```sh
echo 'type=text/plain exts=md,mkd,mkdn,mdwn,mdown,markdown, desc="Markdown document"' >> ~/.mime.types
```

Then restart firefox. Firefox now should render markdown files as html.

-->


## document this installation properly

To document subsequent personal installations of your VM you might wish to update this very file 
you are reading currently.

* In Firefox push the `.md`-Button in the upper right corner of the page
and save the file to `Downloads/build.md`. 
* Alternatively you might want to download the markdown-source of this file locally:

```sh
cd ~/Downloads
wget http://jbusse.de/dsci-lab/_sources/build.md
cd
```

Edit the file `build.md` e.g. with Mousepad. Alternatively you want to use your favourite programming editor.

## VS Code

As of 2024 we recommend *Microsoft Visual Studio Code* (aka VS Code) <https://code.visualstudio.com/docs/python/python-tutorial>:

```sh
snap install code --classic
```

> From File > Preferences > Settings, search for telemetry, and set the Telemetry: Telemetry Level setting to off. This will silence all telemetry events including crash reporting from VS Code. You will need to restart VS Code for the setting change to take effect. <https://code.visualstudio.com/docs/supporting/FAQ#_how-to-disable-crash-reporting>

Optionally also get MyST-Marksown Syntax extension for VS Code: <https://marketplace.visualstudio.com/items?itemName=ExecutableBookProject.myst-highlight>

## vim, emacs

Only if you are over 50 years old ;-) you still want to stick to `vim` or `emacs`:

```sh
sudo apt install vim emacs
```




## More basic Packages

At this stage you have a clean, brand new virtual XUbuntu machine, with VirtualBox Guest extensions enabled.

To get our dsci-lab version you have to install some more packages.  You can do so by simply copying the following commands into a terminal (open a new terminal e.g. by typing `Strg-Alt-t`):


### freeplane

Mindmap (incl. Java):

```sh
sudo apt install freeplane
```

### LaTeX


<https://jupyterbook.org/en/stable/advanced/pdf.html> recommends using the texlive distribution.

We have found that in combination with jupyterbook (see below) the following packes are actually required, with a total size about 2 GB (!):

```sh
sudo apt install imagemagick\
     	 	 texlive-latex-recommended texlive-latex-extra \
                 texlive-fonts-recommended texlive-fonts-extra \
                 texlive-xetex latexmk texlive-lang-german\
		 xindy biber texlive-science

```

<!--
Test the integration of LaTeX installation and jupyterbook:

```sh
cd c
jb build test --builder pdflatex
atril test/_build/latex/book.pdf &
cd
```
-->

### GitHub Desktop

GitHub Desktop - The Linux Fork: <https://github.com/shiftkey/desktop>

```sh
wget -qO - https://apt.packages.shiftkey.dev/gpg.key | gpg --dearmor | sudo tee /usr/share/keyrings/shiftkey-packages.gpg > /dev/null
sudo sh -c 'echo "deb [arch=amd64 signed-by=/usr/share/keyrings/shiftkey-packages.gpg] https://apt.packages.shiftkey.dev/ubuntu/ any main" > /etc/apt/sources.list.d/shiftkey-packages.list'
sudo apt update && sudo apt install github-desktop
```

ca 300 MB (!)



## (Mini-)Conda

Important: *Install Conda not with sudo*. Install it with the role of a normal user instead. Every user and every virtual conda environment are completely independent from each other. There is no system-wide installation.


Installing on Linux:
* https://docs.conda.io/projects/conda/en/stable/user-guide/install/linux.html > Miniconda installer for Linux. > https://docs.anaconda.com/miniconda/ > Linux
  * Miniconda3 Linux 64-bit > ricght click > save to `Downloads/Miniconda3-latest-Linux-x86_64.sh`

```sh
cd
cd Downloads
chmod +x 
```

Miniconda Installer:
<https://docs.conda.io/projects/miniconda/en/latest/> > *Quick command line install*:

```
mkdir -p ~/miniconda3
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh
bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
rm ~/miniconda3/miniconda.sh
~/miniconda3/bin/conda init bash
```

> ==> For changes to take effect, close and re-open your current shell. <==

Why close and open? In an earlier step you have installed `conda`. Conda puts an extra virtual environment layer over the standard Python installation, so we can work with multiple Python configurations in parallel. To learn more about virtual environments:

* <https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-conda.html>
* <https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html>

Your termial command line now should start with `(base)`, which is the name of your current virtual environment:

```
(base) dsci@dsci-lab-ss24:~$ 
```

Keep `conda` current:

```sh
conda update --all
```


(Note:Instead of installing miniconda you instead want to install `anaconda` (<https://www.anaconda.com/products/individual>). Anaconda is much more complete than miniconda, but IMHO fo "fat".  In our dsci-lab we prefer a lightweight system. This allows you to look more easily "under the hood", to understand what's going on, and to maintain the whole system - the dependencies in our setup are complex enough anyhow.)


### Install python packages into the virtual conda environment `base`

As said: Conda is minimalistic. Tus we have to install some modules by ourselfes.

Some important ones are:

```sh
pip install pandas numpy matplotlib scikit-learn seaborn \
   rdflib owlrl markdownify lxml markdown python-slugify \
   jupyter jupyter-book jupytext
```



Notes:

* We do install these packages into the virtual conda environment `base`. If we decide to create another virtual conda environment, it will be empty again, and we have to populate it with libraries again. This is the reason (a) why we prefer lightweight environments, and (b) why we want to learn how to install libraries by ourselfes.


### Jupyterbook

<!--
Generate Jupyterbook config file:

```sh
jupyter notebook --generate-config
```

Edit the file `~/.jupyter/jupyter_notebook_config.py`:
* Search for the line `# c.NotebookApp.use_redirect_file = True`
* uncomment it, and change `True` to `False`. You should get

> c.NotebookApp.use_redirect_file = False
-->


Test the jupyterbook installation: Build the book according to <https://jupyterbook.org/start/build.html>

```sh
mkdir -p ~/c
cd ~/c
jb create jupyterbook-test
```

build html:

```sh
jb build jupyterbook-test
firefox jupyterbook-test/_build/html/index.html &
```

build pdf via LaTeX:

```sh
jb build jupyterbook-test --builder pdflatex
atril jupyterbook-test/_build/latex/book.pdf &
```


<!--
NEW since Ubuntu 22: Firefox is installed via snap. To start juypter we have to set additional environment variables:

```sh
export JUPYTER_DATA_DIR=/home/$USER/Public
export JUPYTER_RUNTIME_DIR=/home/$USER/Public
```

To set these variables permanently add them to the file `.profile`. This file will be executed the next tite you start the display manager. To execute `.profile` in a running display manager (i.e. this bash-session) simply source it:

```sh
source .profile
```
-->

Start Jupyter. Jupyter will launch firefox. Stop Jupyter again:

```sh
jupyter notebook &
jupyter notebook stop
```

<!--
# Add new non-privileged user

```sh
sudo adduser dsci 
sudo addgroup vboxsf
sudo adduser dsci vboxsf
```

Log out. Log in as user dsci.

To continue installation see {doc}`build_dsci`.
-->



## system login screen image

Download desired image to `/Downloads`. 

cp this image with sudo rights into folder `/usr/share/xfce4/backdrops/`:

```
cd  /usr/share/xfce4/backdrops/
sudo mv /tmp/IMG_2247.JPG .
sudo chmod 655 IMG_2247.JPG 
```

"Bildschirmhintergrund des Nutzers verwenden, falls verfügbar": NO


Open Whisker-Menue > Settings > LightDM GTK+ Greeter
* select image in folder `backdrops`
* enter install password
* select background image

## customize user install background image

copy private background image into folder `Documents`.

right click on background > desktop settings > folder Documents > select image

## Zotero

https://www.jbusse.de/dsci-lab/h_zotero.html




# Export OVA

Maschine aktualisieren, Gasterweiterungen installieren, Shared Folder deaktivieren, Medium auswerfen, Maschine 'runterfahren, USB auf 1.0 umstellen, dann *Export OVA*:
* /home/jb/v/dsci-lab_22.04.5_ws24.ova
* "alle Netzwerkadressen mit einbeziehen"
* Hersteller-URL: jbusse.de/dsci-lab

SHA und Größe der Datei feststellen:

```
ls -ls v/dsci-lab_22.04.5_ws24.ova
# 11558000 -rw------- 1 jb jb 11835387904 Sep 30 09:20 v/dsci-lab_22.04.5_ws24.ova

sha256sum v/dsci-lab_22.04.5_ws24.ova
# XXXXXf810530ff34b1a52d6b79f96cb313b8935ccfebd8dcbf56e7868e232cf5fb0a4  v/dsci-lab_22.04.5_ws24.ova
```

Auf den Server hochladen, auch dort `sha256sum` berechnen, kontrollieren, erst dann in {doc}`t_download` eintragen.




# Admin


Aktualisieren:

```sh
sudo apt update; sudo apt upgrade
sudo snap refresh
```

## next steps

<http://jbusse.de/dsci-lab/h_shared-folders.html>

<http://jbusse.de/dsci-lab/h_customizing.html>