The art of data science and scientific computing 
 by Mehmet Süzen 
See also: Science Memo
Preamble The idea of generating randomness via computers goes back to von Neumann's and his team of colleagues work in Princeton and Los Alamos. Even though, he was sceptical about the idea of generating randomness via computer programs, this practice is now almost a de-facto standard, so the name pseudo-random-number-generator (PRNG). Is there a way to generate better PRNGs? This is a continuous…
Preamble Image: Babylonian table for computation (Wikipedia) Probably, one of the most important aspects of computational work in quantitative fields, such as physics and data sciences is stability of numerical computations. It implies given inputs, outputs should not wildly deviates to large numbers or it must not distort the results, such as ranking based on scores, one of the most used…
Rubik's cube (Wikipedia) Preamble Counting is one of the most important concepts in probability and statistical mechanics as well. Two primary characteristics of choosing $n$ items from $N$ items are (no)-order and (no)-repeat . This leads to four possible cases that leads to combinations and permutations of different kinds to found the resulting set size. Specially the case for repeated…
Extension of this post, appear as a short article with a conjecture: Mehmet Süzen. In-context learning as a new kind of symbolic-AutoML: Lyapunov conjecture for CoTs. 2024 HAL France Preamble A graph path (Wikipedia) With the common usage of Pre-trained Large Language Models (PLMs/LLMs), now it is possible to direct them to make data analysis, generate predictions or code for very specialised…
Preamble Probably, one of the most fundamental issue in classical statistical mechanics is extending reversible dynamics to many-particle systems that behaves irreversibly. In other words, how time's arrow appears even though constituted systems evolves in reversible dynamics. This is the main idea of Loschmidt's paradox . The resolution to this paradox lies into something called interventional…
Preamble Figure: Visual description of random stream chunking, M.Suzen (2017) The advent of using computational sciences approaches, i.e., data science and machine learning, in the industry becomes more common practice in almost all organisations due to the democratisation of data science tools and availability of inexpensive cloud infrastructure. This brings the requirement or even compulsory…
Roulette Wheel (Wikipedia) Preamble In this short exposition, we inquire about the purpose of randomness and how this related to discovering or testing causal inferential problem solving using data and causal models. In his seminal work by Holland ( 1986 ) point out something striking that was not put in such form earlier works. He stated the "obvious" that almost all data sets addressing…
Dominos in motion (Wikipedia) Preamble The definition of weighted directed graphs (wDAGs) provides a great opportunity to express causal relationships among given variates. Usually this is expressed as SCMs, Structural Causal Model or in more generally causal model. A given causal model can be expressed as set of simultaneous equations, given a direction for the equality, right to left , meaning…
Preamble The entropy is probably one of the most difficult physical concepts to grasp. Its inception roots in efficiency of engines and foundational connection to multi-particle classical mechanics to thermodynamics, i.e., kinetic theory to thermo-statistics. However, computing entropy for a physical systems is a difficult task, as most of the real-physical systems lacks the explicit formulation.…
Preamble Fractal Tree (Wikipedia) A quiet causal inference revolution is underway in industry. We see immense success of transformers deep learning architectures . However their success should also be attributed to causal modelling. Large Language Models (LLMs) , specially closed-sourced ones, elevates their performance with encoding causal mechanism with human designed deep learning components,…
Kindly reposted to KDnuggets by Gregory Piatetsky-Shapiro Preamble Time-series analysis is needed almost in any quantitative field and real-life systems that collects data over time, i.e., temporal datasets. Building predictive models on temporal datasets for future evolution of systems in consideration are usually called forecasting . The validation of such models deviates from the standard…
Domino OR-gate (Wikipedia) Preamble A trained model may provide predictions about input values it may never seen before but it isn't an inference, at least for 'classical' supervised learning. In reality it provides an interpolation from the training-set, i.e., via function approximation: Interpolation doesn't mean to have all predictions within convex-hull of the training set but interpolation as…
Summary Problem-solving is the core activity of data science using scientific principles and evidence. On our side, there is an irresistible urge to solve the most generic form of the problem. We do this almost always from programming to formulation of the problem. But, don't try to solve a generalised version of the problem. Solve it for N=1 if N is 1 in your setting, not for any integer: Save…
Preamble A recently shown phenomenon can classify deep learning architectures with only using the knowledge gained by trained weights [ suezen20a ]. The classification produces a measure of equivalence between two trained neural network and astonishingly captures a family of closely related architectures as equivalent within a given accuracy. In this post, we will look into this from a conceptual…
Preamble Catalan Castellers are collaborating (Wikipedia) Availability of distributed code tracking tools and associated collaborative tools make life much easier in building collaborative scientific tools and products. This is now especially much more important in data science as it is applied in many different industries as a de-facto standard. Essentially a computational science field in…
Preamble Artificial Intelligence Engines: An introduction to the Mathematics of Deep Learning by Dr James V. Stone the book and Github repository. (c) 2019 Sebtel Press Deep learning and associated connectionist approaches are now applied routinely in industry and academic research from image analysis to natural language processing and areas as cool as reinforcement learning . As practitioners, we…
Summary Many data scientist are trained or experienced in using tools to do statistical modelling, forecasting or machine learning solutions, this doesn't necessarily mean that they should just jump out and do an ad-hoc analysis on the available public data on the covid19 outbreak and draw policy conclusions and publish them in their blogs or the other medium. Rule of thumb of doing such thing you…
Kindly reposted to KDnuggets by Gregory Piatetsky-Shapiro Preamble The core application of machine learning models is a binary classification task . This appears in polyhedra of areas from medicine for diagnostic tests to credit risk decision making for consumers. Techniques in building classifiers vary from simple decision trees to logistic regression and lately super cool deep learning models…
Kindly reposted to KDnuggets by Gregory Piatetsky-Shapiro with the title Applying Occam's razor to Deep Learning Kindly reviewed by Cornelius Weber Preamble: Changing concepts in machine learning due to deep learning Occam's razor or principle of parsimony has been the guiding principle in statistical model selection. In comparing two models, which they provide similar predictions or description…
Fibonacci, Leonardo da Pisa (Wikipedia) Updated with memoization on 26 Feb 2020 Preamble The Fibonacci sequence $F_{n} = F_{n-1} + F_{n-2}$ is famous for both artistic and pure mathematical curiosity as an integer sequence. A simple implementation of this sequence appears as a standard programmer's entry-level job question, along with finding factorial and greatest common divisor. In this short…
As mentioned by Professor Pearl graciously on twitter Preamble Hubble Space Telescope (Wikipedia) Computational science is to modern data scientist, as telescopes are for astrophysics. A better software developer than a statistician and better statistician than a software developer would have been a good definition for the early 2010s in identifying who would be a data scientist. In the late…
Preamble One of the main tenants of practical data science is performing statistical inference on data sets which are assumed to be representations of populations of activity, natural or man-made. A specific case is called causal inference , which probably the core interest of decision makers and probably the main reason why businesses or industry funds data science projects in the first place.…
Kindly reposto to KDnuggets by Gregory Piatetsky-Shapiro Preamble Almost all businesses and industry embraced Machine learning (ML) technologies. Apart from ROI concerns, as it is an expensive endeavour to develop and deploy a service driven by ML techniques, sustainability as in going beyond proof-of-concept core development appears to be one of the roadblocks in data science. In this post, we…
Preamble Ebbinghaus (Wikipedia Machine Learning (ML) is now a de-facto skill for every quantitative job and almost every industry embraced it, even though fundamentals of the field is not new at all. However, what does it mean to teach to a machine? Unfortunately, for even moderate technical people coming from different backgrounds, answer to this question is not apparent in the first instance.…
Kindly reposted to KDnuggets by Gregory Piatetsky-Shapiro with the title Understanding overfitting: an inaccurate meme in machine learning Preamble There is a lot of confusion among practitioners regarding the concept of overfitting . It seems like, a kind of an urban legend or a meme, a folklore is circulating in data science or allied fields with the following statement: Applying…