Package 'DTMCPack'

Title: Suite of Functions Related to Discrete-Time Discrete-State Markov Chains
Description: A series of functions which aid in both simulating and determining the properties of finite, discrete-time, discrete state markov chains. Two functions (DTMC, MultDTMC) produce n iterations of a Markov Chain(s) based on transition probabilities and an initial distribution. The function FPTime determines the first passage time into each state. The function statdistr determines the stationary distribution of a Markov Chain.
Authors: William Nicholson
Maintainer: William Nicholson<[email protected]>
License: GPL (>= 2)
Version: 0.1-3
Built: 2025-02-10 02:57:20 UTC
Source: https://github.com/cran/DTMCPack

Help Index


Suite of functions related to discrete-time discrete-state Markov Chains

Description

A series of functions which aid in both simulating and determining the properties of finite, discrete-time, discrete state markov chains. This package may be of use to practioners who need to simulate Markov Chains, but its primary intended audience is students of an introductory stochastic processes studying class properties and long run behavior patterns of Markov Chains. Two functions (DTMC, MultDTMC) produce n iterations of a Markov Chain(s) based on transition probabilities and an initial distribution. The function FPTime determines the first passage time into each state. The function statdistr determines the stationary distribution of a Markov Chain. Updated 4/10/22 to maintain compatibility with R.

Details

Package: DTMCPack
Type: Package
Version: 0.1-2
Date: 2013-05-22
License: GPL(>=2)
LazyLoad: yes

Author(s)

Will Nicholson

Maintainer: <[email protected]>

References

Sidney Resnick, "Adventures in Stochastic Processes"

Examples

data(gr)
data(id)
DTMC(gr,id,10,trace=FALSE)

Simulation of Discrete-Time/State Markov Chain

Description

This function simulates iterations through a discrete time Markov Chain. A Markov Chain is a discrete Markov Process with a state space that usually consists of positive integers. The advantage of a Markov process in a stochastic modeling context is that conditional dependencies over time are manageable because the probabilistic future of the process depends only on the present state, not the past. Therefore, if we specify an initial distribution as well as a transition matrix, we can simulate many periods into the future without any further information. Future transition probabilities can be computed by raising the transition matrix to higher-and higher powers, but this method is not numerically tractable for large matrices. My method uses a uniform random variable to iterate a user-specified number of iterations of a Markov Chain based on the transition probabilities and the initital distribution. A graphical output is also available in the form of a trace plot.

Usage

DTMC(tmat, io, N, trace)

Arguments

tmat

Transition matrix-rows must sum to 1 and the number of rows and columns must be equal.

io

Initial observation, 1 column, must sum to 1, must be the same length as transition matrix.

N

Number of simulations.

trace

Optional trace plot, specify as TRUE or FALSE.

Value

Trace

Trace-plot of the iterations through states (if selected)

State

An n x nrow(tmat) matrix detailing the iterations through each state of the Markov Chain

Author(s)

Will Nicholson

References

"Adventures in Stochastic Processes" by Sidney Resnick

See Also

MultDTMC

Examples

data(gr)
data(id)
DTMC(gr,id,10,trace=TRUE) 
# 10 iterations through "Gambler's ruin"

First Passage Time

Description

This function uses the companion function multDTMC to simulate several Markov chains to determine the first passage time into each state, i.e. the first time (after the initial iteration) that a specified state is reached in the Markov Process. First Passage Time can be useful for both determining class properties as well as the stationary/invariant distribution for large Markov Chains in which explicit matrix inversion is not computationally tractable.

Usage

FPTime(state, nchains, tmat, io, n)

Arguments

state

State in which you want to find the first passage time.

nchains

Number of chains you wish to simulate.

tmat

Transition Matrix, must be a square matrix, rows must sum to 1.

io

Initial Distribution

n

Number of iterations to run for each Markov Chain.

Value

fp1

Vector of length(nchains) which gives first passage time into the specified state for each Markov Chain.

Author(s)

Will Nicholson

See Also

DTMC

Examples

data(gr)
data(id)
FPTime(1,10,gr,id,10) # First passage time into first state on Gambler's ruin

Example Data Set: Gambler's ruin on 4 states

Description

Motivating example, random walk with absorbing boundaries on 4 states. Analogous to a gambler at a casino. The 4 states represent a range of wealth. States 1 and 4 are absorbing with state 1="Broke", state 4="wealthy enough to walk away" and the intermediate states 2 and 3 are transitory. It is assumed that he bets of all his winnings in the intermediate states and has equal probability of winning and losing

Examples

data(gr)
data(id)
DTMC(gr,id,10,trace=FALSE)

Harry the SemiPro

Description

Example Markov Chain from page 139 of Resnick. The protagonist, basketball player "Happy Harry's" productivity fluctuates between three states (0-1 points), (2-5 points), (5 or more points) and the transition between states can be modeled using a Markov Chain. Used as a motivating example to calculate the long run proportion of time spent in each state using the statdist function.

Source

Sidney Resnick "Adventures in Stochastic Processes"

Examples

data(hh)
statdistr(hh)

Initial distribution

Description

A starting distribution for the gambler's ruin example, which assigns equal probability of starting in each state.

Examples

data(id)
data(gr)
DTMC(gr,id,10,trace=FALSE)

Multiple Discrete time Markov Chains

Description

An extension of the DTMC package which enables multiple cocurrent Markov Chain simulations. At this time, plotting is not enabled.

Usage

MultDTMC(nchains, tmat, io, n)

Arguments

nchains

Number of chains to simulate (integer).

tmat

Transition Matrix

io

Initial distribution

n

Number of iterations to run each chain.

Value

chains

Returns nchains matrices of length nrow(tmat) by n which depict the transition of the Markov Chain.

Author(s)

Will Nicholson

See Also

DTMC

Examples

data(gr)
data(id)
MultDTMC(20,gr,id,10) # 20 chains with 10 iterations using the Gambler's ruin example.

Computing Stationary Distribution

Description

This function computes the stationary distribution of a markov chain (assuming one exists) using the formula from proposition 2.14.1 of Resnick: pi=(1,...1)(I-P+ONE)^(-1), where I is an mxm identity matrix, P is an mxm transition matrix, and ONE is an mxm matrix whose entries are all 1. This formula works well if the number of states is small, but since it directly computes the inverse of the matrix, it is not tractable for larger matrices. For larger matrices 1/E(FPTime(n)) is a rough approximation for the long run proportion of time spent in a state n.

Usage

statdistr(tmat)

Arguments

tmat

Markov chain transition matrix, must be a square matrix and rows must sum to 1.

Value

Returns a stationary distribution: mxm matrix which represents the long run percentage of time spent in each state.

Author(s)

Will Nicholson

References

Resnick, "Adventures in Stochastic Processes"

Examples

data(hh)
statdistr(hh)