build the dsci-lab by yourself¶
This file:
Build of
xubuntu_dsci_ss22
, 2022-02-25Xubuntu 20.04.3 LTS (Focal Fossa), 64-bit
Important: You are not requested to build the dsci-lab by yourself. Instead you are advised to download the fully configured dsci-lab as a ready-to-use virtual machine from http://jbusse.de/dsci-lab/.
However, if you are a lecturer or a deeply interested student you might want to learn how we set up the dsci-lab. Here are the steps.
Install VM Virtual Box 6.1¶
https://www.oracle.com/de/virtualization/technologies/vm/downloads/virtualbox-downloads.html
2022-02-25: The latest release is version 6.1.32
Start VirtualBox. Also install the Oracle VM VirtualBox Extension Pack:
push link 6.1.32 ExtPack, select open with Oracle VM Virtual Box (Standard)
Create the virtual XUbuntu machine¶
Download an ISO image (ca. 1.8 GB) of the LTS release: 20.04, Focal Fossa, 64-bit of XUbuntu
e.g. from https://xubuntu.org/download/ > Germany
file: http://ftp.uni-kl.de/pub/linux/ubuntu-dvd/xubuntu/releases/20.04/release/xubuntu-20.04.3-desktop-amd64.iso (2021-08-19 12:57, 1.7G): right click on file, “save as”, save to disk (e.g.
/home/user/v/
)
Start Oracle VM Virtual Box Manager
Maschine > Neu > Name: e.g. xubuntu_dsci_ss22, Typ: Linux, Version: Ubuntu (64-bit)
Speichergröße: 4096 MB
Platte: Festplatte erzeugen > Erzeugen
Dateityp der Festplatte: VDI
Art der Speicherung: dynamisch alloziert
Dateiname und Größe: hier mindestens 100 GB angeben > Erzeugen
Start machine xubuntu_dsci_ws21. A window will pop up: “Medium für Start auswählen”
Medium hinzufügen > locate xubuntu-20.04.3-desktop-amd64.iso on your disc (ca. 1.8 GB) > Auswählen
Starten
VirtualBox will start the ISO image:
Select language, e.g. “Deutsch” > “Install XUbuntu”
German > German
Aktualisierungen herunterladen
Installationsart: Festplatte löschen und installieren (this will clean only your newly allocated virtual hard disc, NOT the disk of your host system); > Installieren
Wo befinden Sie sich? > Berlin
“Who are You?”:
Name: Data Scientist
Name des Rechners: dsci-ss2022
Benutzername: dsci
password: dscidsci (this is an intitial and very weak password, you MUST it change later!)
A basic version of xubuntu will be installed.
Install Basic System¶
“Die Installation ist abgeschlossen. Sie müssen jetzt den Rechner neu starten, um das System zu benutzen” > Jetzt neu starten
“Remove installation medium”: (nothing to do), “press ENTER”: press Enter!
log in with user dsci (“Data Scientist”), password dscidsci
update system¶
“Aktualisierungsverwaltung” will start automatically and ask you to update the system: Do it, “jetzt installieren”.
To start Aktualisierungsverwaltung manually:
Whisker-Menu > Einstellungen > Aktualisierungsverwaltung
To update your installation, regularly start “Aktualisierungsverwaltung”.
Restart the machine afterwards. Login again.
We will use the command line hereafter where possible. Get a new terminal by typing Strg-Alt-t
.
Instead of using “Aktualisierungsverwaltung” you can keep the system current with apt
:
sudo apt update; sudo apt upgrade
Install Guest Extensions¶
To prepare the installation of new kernel modules we need gcc:
sudo apt install gcc make perl
Oracle Virtual Box > Geräte > Gasterweiterungen einlegen: A window pops up, showing the directory /media/data/VBox_GAs_6.1.32./
> rightclick on background, “Terminal hier öffnen”, a new terminal opens. Type in:
sudo ./VBoxLinuxAdditions.run
Reboot the VM:
sudo reboot
Login again. You now should be able to resize the VM window!
Activate bidirectional Copy & Paste from Windows-Host to VB etc.
Geräte > Gemeinsame Zwischenablage > bidirektional
Configure Firefox¶
Start “Internetnavigator”, i.e. Firefox. Navigate to http://jbusse.de/dsci-lab/dsci-lab-build.html. Alternatively manually type in (and don’t forget to add the ampersand “&” at the end of the line):
firefox http://jbusse.de/dsci-lab/dsci-lab-build.html &
If your WLAN works properly, Firefox will open this page in your new linux machine. This allows you to copy & paste the commands below to your terminal. Bookmark this page in Firefox by hitting CTRL-D.
Firefox Privacy¶
start firefox. Hit the “ALT”-button to show the menu.
Bearbeiten > Einstellungen
Startseite und neue Fenster: point default to
https://www.startpage.com/
Inhalte des Firefox-Startbildschirms: privacy! … but do not deactivate these widges (don’t close your eyes), but go to
Datenschutz und Sicherheit > Benuzerdefiniert > Cookies und Website-Daten
Cookies und Website-Daten beim Beenden von Firefox löschen: Check!
Hauptpasswort verwenden: Enter a save password!
Chronik: Firefox wird eine Chronik > nach benutzerdefinierten Einstellungen anlegen
“Die Chronik löschen, wenn Firefox geschlossen wird”: Check!
Datenerhebung durch Firefox und deren Verwendung
(de-)select whatever you want
Extras > Add-Ons und Themes > Erweiterungen
Ghostery – datenschutzorientierter Werbeblocker
View markdown with Firefox¶
(1) install Add-On: Extras > Add-Ons und Themes > Erweiterungen
Markdown Viewer Webext von Kulero, Cimbali
(2) add MIME type: https://github.com/KeithLRobertson/markdown-viewer#support-for-local-files-on-linux:
Firefox on Linux may not know how to handle markdown files by default (see #2). There are a number of possible workarounds for this (see this SuperUser question for example). […] Another workaround (which might cover other OSs as well), is to edit Firefox’s private mime types. These mime types are stored in a file indicated by helpers.private_mime_types_file, by default it is ~/.mime.types
:
echo 'type=text/plain exts=md,mkd,mkdn,mdwn,mdown,markdown, desc="Markdown document"' >> ~/.mime.types
Then restart firefox. Firefox now should render markdown files as html.
Update this file?¶
To document subsequent personal installations of your VM you might wish to update this very file you are reading currently.
In Firefox push the
.md
-Button in the upper right corner of the page and save the file toDownloads/dsci-lab-build.md
.alternatively you might want to download the markdown-source of this file locally:
cd ~/Downloads
wget http://jbusse.de/dsci-lab/_sources/dsci-lab-build.md
cd
Edit the file dsci-lab-build.md
e.g. with Mousepad. Alternatively you want to use your favourite programming editor, e.g.
sudo apt install emacs
sudo apt install vim
Basic Packages¶
At this stage you have a clean, brand new virtual XUbuntu machine, with VirtualBox Guest extensions enabled.
To get our dsci-lab version you have to install some more packages. You can do so by simply copying the following commands into a terminal (open a new terminal e.g. by typing Strg-Alt-t
):
Mindmap (incl. Java):
sudo apt install freeplane
Conda, Jupyter, Jupyterbook¶
sudo apt install jupyter
Get Miniconda from https://docs.conda.io/en/latest/miniconda.html#latest-miniconda-installer-links:
cd Downloads/
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
chmod +x ./Miniconda3-latest-Linux-x86_64.sh
./Miniconda3-latest-Linux-x86_64.sh
Do you accept the licence terms? yes
Do you wish the installer … by running conda init? yes
(Instead of installing miniconda you also might want to install anaconda
(https://www.anaconda.com/products/individual). Anaconda is much more complete than miniconda, but IMHO fo “fat”. In our dsci-lab we prefer a lightweight system. This allows you to look more easily “under the hood”, to understand what’s going on, and to maintain the whole system - the dependencies in our setup are complex enough anyhow.)
After you have installed Conda, close your terminal (CTRL-D
) and open a new terminal again (e.g. with Strg-Alt-t
). (Why close and open? In an earlier step you have installed conda
. Conda puts an extra virtual environment layer over the standard Python installation, so we can work with multiple Python configurations in parallel. To learn more about conda virtual environments see https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-conda.html)
Your termial command line now should start with (base)
, which is the name of your current virtual conda environment:
(base) jb@dsci-vbox:~$
As said: Conda is minimalistic. Tus we have to install some modules by ourselfes. Some important ones are:
conda install pandas numpy matplotlib scikit-learn seaborn xgboost lxml markdown python-slugify
SPÄTER NOCH INSTALLIEREN:
# pip install rdflib owlrl markdownify
Notes:
We do install these packages into the virtual conda environment
base
. If we decide to create another virtual conda environment, it will be empty again, and we have to populate it with libraries again. This is the reason (a) why we prefer lightweight environments, and (b) why we want to learn how to install libraries by ourselfes.Caveat: Install Conda not with sudo, but instead with the role of a normal user. Every user and every virtual conda environment are completely independent from each other. There is no system-wide installation.
Keep conda
current:
conda update --all
Jupyterbook:
conda install -c conda-forge jupyter-book
Jupytext is already installed then:
conda install -c conda-forge jupytext
Test the installation: Build the book according to https://jupyterbook.org/start/build.html
mkdir -p ~/b
cd ~/b
jb create test
jb build test
cd
LaTeX¶
https://jupyterbook.org/advanced/pdf.html recommends using the texlive distribution (https://www.tug.org/texlive/), size about 1.9 GB (!):
sudo apt install imagemagick
sudo apt-get install texlive-latex-recommended texlive-latex-extra \
texlive-fonts-recommended texlive-fonts-extra \
texlive-xetex latexmk
Test the integration of LaTeX installation and jupyterbook:
cd b
jb build mybookname/ --builder pdflatex
jb build test --builder pdflatex
atril test/_build/latex/book.pdf &
cd
spaCy¶
Linux, X86, conda, CPU
NO virtual env
Trained pipelines: English, German
conda install -c conda-forge spacy
python -m spacy download en_core_web_sm
python -m spacy download de_core_news_sm
Zotero¶
Goto Zotero https://www.zotero.org/download/ > installations help > Linux > Debian/Ubuntu-based Distros > “A longtime community member maintains zotero-deb”
wget -qO- https://apt.retorque.re/file/zotero-apt/install.sh | sudo bash
sudo apt update
sudo apt install zotero
Also do install the Zotero Firefox Connector from the zotero-site: https://www.zotero.org/download/ > Install Firefox connector.
Launch Zotero the first time.
zotero
Zotero wants to install severals defaults: say yes!
Zotero’s Better BibTeX¶
Enhance Zotero with https://retorque.re/zotero-better-bibtex/: Follow the installation instructions from https://retorque.re/zotero-better-bibtex/installation/ > latest release:
Save file
zotero-better-bibtex-6.2.9.xpi
e.g. toDownloads
…if not alread running: start zotero
and import this file: Zotero > Werkzeuge > Add-ons > Install Add-on from File > select
zotero-better-bibtex-6.2.9.xpi
>
Zotero wants you to restart Zotero two times: do it. You will find that Zotero has allocated the new directory ~/Zotero
: Include this directory to your list of directories which are backuped daily ;-)
Learn where your data reside: https://www.zotero.org/support/zotero_data.
Export OVA¶
Maschine aktualisieren, Gasterweiterungen installieren, Medium auswerfen, Maschine ‘runterfahren, USB auf 1.0 umstellen, dann Export OVA:
/home/jb/v/xubuntu_dsci_ss22.ova
alle Netzwerkadressen mit einbeziehen
Hersteller-URL: