Decision trees & gradient boosting
A decision tree splits the data with a sequence of yes/no questions (“heart rate > 120?”) until each leaf holds one prediction. Single trees are easy to read but unstable. Gradient boosting (XGBoost, LightGBM) builds many small trees in sequence, where each new tree is fit to the errors the current ensemble still makes. The result is one of the strongest models for tabular data and a common baseline in my classification experiments.
References: Breiman et al., Classification and Regression Trees (Wadsworth, 1984); Friedman, “Greedy function approximation: a gradient boosting machine,” Annals of Statistics 29(5), 2001; Chen & Guestrin, “XGBoost,” KDD 2016 — arXiv:1603.02754.
Neural networks (MLP) & backpropagation
A multilayer perceptron stacks layers of the form output = activation(W·input + b). The non-linear activation (ReLU, etc.) is what lets stacked layers represent complex functions. Training means measuring a loss, then using backpropagation — the chain rule applied layer by layer — to get the gradient of that loss with respect to every weight, and nudging the weights against it.
References: Rumelhart, Hinton & Williams, “Learning representations by back-propagating errors,” Nature 323, 1986; Goodfellow, Bengio & Courville, Deep Learning (MIT Press, 2016), ch. 6 — deeplearningbook.org.
LSTM networks
A recurrent network reads a sequence one step at a time, carrying a hidden state forward. Plain RNNs forget quickly because gradients shrink over many steps. An LSTM adds a separate cell state and three gates (input, forget, output) that learn what to keep, overwrite, or expose. That makes it good at the timing and trends in vital-sign streams, which is why it appears in both my imputation and assistant-system work.
References: Hochreiter & Schmidhuber, “Long short-term memory,” Neural Computation 9(8), 1997 — doi:10.1162/neco.1997.9.8.1735.
Autoencoders & denoising autoencoders
An autoencoder compresses an input to a small code and reconstructs it, learning the data’s structure in the bottleneck. A denoising autoencoder is trained to rebuild a clean input from a corrupted one, which makes its features more robust and turns it into a natural imputer: mark missing values as corruption and read off the reconstruction.
References: Vincent, Larochelle, Bengio & Manzagol, “Extracting and composing robust features with denoising autoencoders,” ICML 2008; Vincent et al., “Stacked denoising autoencoders,” JMLR 11, 2010 — jmlr.org/papers/v11/vincent10a.
Generative adversarial networks (GANs)
Two networks play a game. The generator turns random noise into fake samples; the discriminator tries to tell fake from real. Training the two together pushes the generator toward samples the discriminator cannot distinguish from the real distribution. GANs are powerful but delicate — unstable training and mode collapse (the generator producing only a few kinds of sample) are the usual failure modes, and much of tabular-GAN research is about avoiding them.
References: Goodfellow et al., “Generative adversarial nets,” NeurIPS 2014 — arXiv:1406.2661; Salimans et al., “Improved techniques for training GANs,” NeurIPS 2016 — arXiv:1606.03498.
Attention & transformers
Attention lets a model, for each position in a sequence, take a weighted average of all other positions, with the weights computed from how relevant each one is. Stacking multi-head attention with feed-forward layers and positional encodings gives the transformer — parallel to train, good at long-range structure, and the standard encoder for the free-text part of a multimodal clinical model.
References: Bahdanau, Cho & Bengio, “Neural machine translation by jointly learning to align and translate,” ICLR 2015 — arXiv:1409.0473; Vaswani et al., “Attention is all you need,” NeurIPS 2017 — arXiv:1706.03762.
Multimodal fusion
When inputs come in different forms — signals, tables, text — you can combine them early (concatenate raw features), late (train one model per modality and merge predictions), or jointly (encode each modality, then fuse the encodings and train end to end). The TeleNotArzt model uses joint fusion so that a weak or missing modality is down-weighted rather than breaking the pipeline.
References: Baltrušaitis, Ahuja & Morency, “Multimodal machine learning: a survey and taxonomy,” IEEE TPAMI 41(2), 2019 — arXiv:1705.09406.
Ranking metrics (recall@k, coverage)
When a model returns a ranked shortlist rather than one label, accuracy is the wrong lens. recall@k asks whether the correct item is in the top k. coverage asks whether, across a case, every action that should have been suggested was suggested somewhere in the lists. Together they measure both “is the answer near the top” and “did we miss anything”.
References: Manning, Raghavan & Schütze, Introduction to Information Retrieval (Cambridge, 2008), ch. 8 — nlp.stanford.edu/IR-book. Coverage as used here is defined in my TeleNotArzt paper — doi:10.5220/0014481000004052.