Introduction to Stochastic Modeling

(3 Credit Hours) · Undergraduate

Stochastic modeling is a technique of presenting data or predicting outcomes that takes into account a certain degree of randomness, or unpredictability. Topics include (i) Short Review of Probability - Major discrete and continuous distributions, properties of random variables. (ii) Conditional probability and conditional expectation, sums of random variables, martingales. (iii) Introduction to Discrete Markov Chains - Transition probability matrix of a Markov chain, some Markov chain models, first step analysis, the absorbing Markov chains, various types and classifications of Markov chains. (iv) Long Run (asymptotic) Behavior of Markov Chains: Limiting distribution, the classification of states, irreducible Markov chains, periodicity of Markov chains, recurrent and transient states, the basic limit theorem of Markov chains. (v) Poisson Processes - The Poisson distribution and the Poisson process, the law of rare events, distributions associated with the Poisson process, the Uniform distribution and Poisson processes. (vi) Continuous Time Markov Chains - Pure birth and death processes and it's limiting behavior. (vii) Introduction to Brownian Motion, Drift and Diffusion, Geometric Brownian motion, Ornstein-Uhlenbeck process and it's long run behavior. (viii) Monte Carlo Simulations for Diffusion.