ML context

domain of application: high-dimensional, broad datasets (p columns >> n rows)

• bag of words, tf-idf, classification
• document retrieval, search query expansion
• Internet of Things (IOT), sensor data

interesting ML algorithms: https://scikit-learn.org/stable/modules/linear_model.html:

• 1.1.2. Ridge Regression
• 1.1.3. Lasso | 1.1.8. LARS Lasso
• 1.1.5. Elastic Net
  • regression only, NO logit version for classification available in scikit :-(
  • but see below, SVM-version
• 1.1.11. Logistic regression
  • The solver "liblinear" uses a coordinate descent (CD) algorithm, and relies on the excellent C++ LIBLINEAR library ...
  • The solvers implemented in the class LogisticRegression are "liblinear", "newton-cg", "lbfgs", "sag" and "saga"
• 1.1.12. Stochastic Gradient Descent - SGD

my personal favourite: ElasticNet

• explicitly suitable for bag of word models with p >> n and highly correlated attributes
• fast fit: highly optimized, very fast algorithm glmnet in R
• uses a sequence of warm starts, thus suitable for online learning
• small whitebox models
• very fast prediction

good old-fashioned idea

the general Q is: How can we improve ML (here: ElasticNet model generation) by adding semantic background knowledge?

The curse praise of dimensionality: Use inferencing to enrich the feature set and to propagate values. Also allow for adding new features (single, married etc.).

first results