There are two direct benefits to training a Markov Chain on data. You can:

1. Analyze the structure of the model to understand the qualities of the data.

2. Generate new data that is very similar to the training data.

The focus of this article series is more the second point, but the first can help with the second. Trying to generate data raises some questions.

• How do you know if the data generated actually is a reasonable facsimile of the training data?

• Are there any inputs to the process, other than the training data, that could influence whether that generated data is a reasonable facsimile?

• Much like with sampling in the field of statistics, how do you know when you have enough data from a Markov Chain?

Prev: Markov Chains with NetworkX and PyDTMC

Generating data from a Markov Chain requires an initial state to begin the process, and a random number each time we take a step. To cover these questions we will:

1. Create runs of output data. These are sequences of states.

2. Analyze the probability distribution of the states in those runs.

3. Make analysis repeatable by controlling random numbers via seeds.

4. Examine sensitivity of the runs to the choice of seed used.

5. Use all the possible initial states as initial states.

6. Assess how long it takes the model to “forget” the state it started with when generating output. We want it to forget because the choice of initial state is somewhat — later we’ll get into why I said “somewhat” — arbitrary, and we don’t want to wire a persistent bias into the generated data.

Some of this can be examined empirically by doing data-generation experiments and analyzing the results. Some can be estimated analytically via known techniques for Markov Chains.

We’ll be doing both. The empirical can help explain the purpose of the analytical.

Taxi Trips as a DTFMC, Revisited

Once again, here is our adjacency matrix that we use to build a Discrete-Time Finite Markov Chain.

adj_rows = dict(
    airport= [  0, 0.2, 0.7, 0.1],
    hospital=[0.4,   0, 0.6,   0],
    hotel=   [0.4, 0.2,   0, 0.4],
    mall=    [0.2, 0.1, 0.7,   0],
)
adj_matrix = pd.DataFrame.from_dict(adj_rows,
                                    orient='index',
                                    dtype=float,
                                    columns=list(adj_rows.keys()))
model = mc.MarkovChain(adj_matrix)

From this model, we can use the analysis tools of the chain to get the stationary distribution, which is the theoretical long-running distribution of states that would appear in chains. We’ll wrap it in a nicer DataFrame to match the adjacency matrix:

data = {
  # In PyDTMC 'stationary_distributions' is an alias of 'pi';
  # You'll see pi symbols in mathematical treatments of Markov Chains
  'probability': model.pi[0]
}
distribution: pd.DataFrame = pd.DataFrame(data=data,
                                          index=adj_matrix.index.copy())
distribution.index.name = 'state'

We can visualize this as a histogram:

distribution.plot(kind='bar', color='mediumorchid')

Going back to our earlier questions:

1. “How do you know if the data generated actually is a reasonable facsimile of the training data?” The answer revolves around deciding if our generated data has a distribution similar enough to the stationary distribution. The generated data should have a histogram roughly like what we see above.

2. “Are there any inputs to the process, other than the training data, that could influence whether that generated data is a reasonable facsimile?” Here the answer will be determining if the initial state or the seed or the number of steps of progress cause the generated data to not look like the stationary distribution.

3. “How do you know when you have enough data from a Markov Chain?” It is possible that some of the concerns of the second question above go away when enough data is generated. We will examine how much is enough for us to see approximately the correct histogram.

Lining Up The Pieces

Piece 1: “Create runs of output data. These are sequences of states.”

PyDTMC gives us a starting point for this. It takes a little tidying to map between state names and state numbers (row offsets in the adjacency matrix):

from_state_mapping =\
    {state: index_pos for index_pos, state in enumerate(adj_matrix.index)}
from_index_pos_mapping =\
    {index_pos: state for index_pos, state in enumerate(adj_matrix.index)}
simulated = model.simulate(steps=5,
                           initial_state=from_state_mapping['hotel'],
                           seed=42,
                           output_indices=True)
states = [from_index_pos_mapping[index_pos] for index_pos in simulated]

That gets us:

The result is our initial state, 'hotel', plus the 5 randomly-generated next states that we asked for.

We will build on that and create a Pandas DataFrame suited to how we will analyze generated runs.

Piece 2: “Analyze the probability distribution of the states in those runs.”

Ok, this one takes a little more coding work. We need to:

• Generate the simulated run.

• Treat each value in that run as a DataFrame row corresponding to a step.

• Convert the string name of the step into separate column, so 'airport' is counted in a different column than 'mall'; a marker value of 1 means present, 0 means absent.

• Make running aggregates at each step, so that at step 3 we have the per-state aggregate from the initial state + step 1 + step 2 + step 3.

• Use those aggregates to generate the probability distribution implied at that step. This means adding the aggregates to get a total number of all states seen, then dividing the aggregates by the total.

Here we go!

# 'states' are from the simulated run above
running_df = pd.DataFrame({'state': states})
running_df.index.name = 'step'
# count how many of each state we used as of each step in the process
state_count_df = pd.get_dummies(data=running_df, 
                                prefix='', 
                                prefix_sep='').cumsum()
# add any missing states because we may not have used them all yet
# and cumsum would drop missing states from the result
model_states = adj_matrix.index.tolist()
missing_columns = set(model_states) - set(state_count_df.columns)
if missing_columns:
    for state in missing_columns:
        state_count_df[state] = 0
        # debugging is easier it we keep the state column-order canonical
        state_count_df = state_count_df[self.model_states]
# add the count aggregates to our running log
running_df = pd.concat([running_df, state_count_df], axis=1)
# turn the current row into its implied distribution
distribution_df = state_count_df.div(state_count_df.sum(axis=1), axis=0)
distributions = [row.to_frame() for _, row in distribution_df.iterrows()]
for distribution in distributions:
    # make it look like the structure of the stationary distribution
    distribution.index.name = 'state'
    distribution.rename(columns={0:'probability'}, inplace=True)
running_df['distribution'] = distributions

The outcome is a running total of the appearances of each state. The distribution column looks a little messy but that is because each value is itself an entire DataFrame object, which will come in handy later.

Piece 3: “Make analysis repeatable by controlling random numbers via seeds.”

We’ll be doing a decent number of runs where we want to extract the signal of what we are attempting from the surrounding noise of randomness. Because of that I decided to use random numbers generated by the Python secrets module so that I would be tapping whatever entropy backed /dev/urandom. The seeds are then saved to a Parquet file for re-use.

class SeedHandler:
    
    def __init__(self, *, num_seeds, bits_per_seed, seed_location):
        self.num_seeds = bits_per_seed
        self.bits_per_seed = bits_per_seed
        self_seed_location = seed_location

    @property
    def seeds(self):
        if not os.path.exists(self.seed_location):
            seed_values = self._make_seeds()
            self._write_seeds(seed_values=seed_values)
        else:
            seed_values = self._read_seeds()
        return seed_values

    def _make_seeds(self):
        values = []
        for i in range(self.num_seeds):
            value = secrets.randbits(self.bits_per_seed)
            values.append(value)
        return tuple(values)

    def _read_seeds(self):
        seed_df = pd.read_parquet(path=self.seed_location,
                                  engine='pyarrow')
        seed_ds = seed_df.astype(dtype=dict(seed=self.numpy_dtype))['seed']
        return tuple(seed_ds)

    def _write_seeds(self, *, seed_values):
        seed_ds = pd.Series(data=seed_values, name='seed')
        seed_df = seed_ds.to_frame()
        seed_df.to_parquet(path=self.seed_location, engine='pyarrow')

There’s more code in the Jupyter notebook, but the above shows the essence.

Piece 4: “Examine sensitivity of the runs to the choice of seed used.”

The resulting seeds will be supplied to every call of simulate(), which will be handled within the function that generates the states and builds running_df. We just need to augment that DataFrame with a 'seed' column.

for seed in seeds:
    running_df =\
        running_distributions(steps=steps,
                              initial_state=initial_state,
                              seed=seed)
        # get a 'step' column from the existing index
        running_df.reset_index(drop=False, inplace=True)
        # and add the seed
        running_df['seed'] = seed

That gets us two out of three pieces of what will become our multi-index.

Piece 5: “Use all the possible initial states as initial states.”

And now we get our third piece of the multi-index. We need another loop around the code above so we can supply different choices of initial_state, add a column to track it, build an index from the three parts, and pd.concat() the running_df from each (initial_state, seed) combination.

index_columns = ['initial_state', 'step', 'seed']
across_states_and_seeds_df = None
for initial_state in model_states:
    for seed in seeds:
        running_df =\
            running_distributions(steps=steps,
                                  initial_state=initial_state,
                                  seed=seed)
            running_df.reset_index(drop=False, inplace=True)
            running_df['initial_state'] = initial_state
            running_df['seed'] = seed
            # create the multi-index
            running_df.set_index(index_columns, drop=True, inplace=True)
            if across_states_and_seeds_df is None:
                across_states_and_seeds_df = running_df
            else:
                across_states_and_seeds_df =\
                    pd.concat([across_states_and_seeds_df, running_df],
                              axis=0)

Piece 6: “Assess how long it takes the model to forget the state it started with.”

This will be examined in two different ways.

1. Analytically.

2. From the data.

The analytical part is supplied by a PyDTMC model property:

model.mixing_rate

I believe this property is misnamed. The implementation appears to be what is usually called the “relaxation rate” or “convergence rate”. This is the rate at which deviations from the stationary distribution shrink as more steps are added to the chain generated. It’s derived from the same eigenvalue information as the spectral gap.

To get an estimate of how many steps that implies — the mixing time — before a desired amount of convergence is achieved, we would have to adjust for how close we want the generated distribution to be to the stationary one.

This is a bit opaque since we aren’t really indicating what we are measuring via that rate, but as a first approximation we’ll use epsilon=0.001:

epsilon = 0.001
taxi_structure.model.mixing_rate * math.log(1 / epsilon)

That’s our analytical reference point, we want to see how the chain distributions look when we have the 19th step after the initial state.

The reason I picked an epsilon that small is because we haven’t discussed what kind of distance should matter. I did some digging via the PyDTMC source code, Wikipedia, and several Monte Carlo texts to get an answer.

• If I were to use the PyDTMC mixing_time() method I believe it uses the square root of the dot product between the stationary distribution and the distribution of the generated chain.

• The more usual mathematical definition is based on the greatest absolute difference between the two distributions, called the “Total Variation Distance” (TVD). It’s definitely the one I find discussed in material on convergence.

The second is closer to what we want, but not quite. We care about:

• Average closeness: what PyDTMC provides.

• Worst-case closeness: what TVD provides.

• Similarity of shape between the two distributions: which neither provide.

We want all three, and getting the third with a small value gets you the first two.

As a reminder of what we’re trying to achieve in this piece, we want to know when the choice of initial state no longer has a meaningful bias on the distribution of the chain. By comparing the distribution of the generated chain to the stationary distribution, when they are similar enough we are declaring that there is no longer any level of bias from the initial state that we would care about.

For this reason I added a different measure of closeness to the code, based upon the Jensen-Shannon Distance (JSD). It is histogram-friendly by being symmetric (comparing A to B is the same as comparing B to A), and robust to empty bins:

eps: float = 1e-9
adjusted_sample = (sample + eps) / (sample + eps).sum()
distance = sp_sd.jensenshannon(p=stationary_distribution,
                               q=adjusted_sample)[0]

The adjustment shown deals with a limitation in JSD when the two distributions get very close and you’re faced with the limit of dividing a small value by another value that is approaching zero.

There is a bit more code to getting the distance behaving when faced with numerical error, which is in the Jupyter notebook. It’s a little bit of a hack but the alternative would be to implement JSD from scratch to deal with a special case where numerical error triggered sqrt(negative number) instead of sqrt(zero). I found in practice a small number of random seeds tripped over that case when generating large sequences.

Assembling the Pieces

Now that we have a notion of distance, we need to add it to our computation of the running results.

We will want to examine that distance at each step and see if it is mostly declining, and what our measure is at the 19th step we derived analytically.

# 'states' are from the simulated run above
running_df = pd.DataFrame({'state': states})
running_df.index.name = 'step'
# count how many of each state we used as of each step in the process
state_count_df = pd.get_dummies(data=running_df, 
                                prefix='', 
                                prefix_sep='').cumsum()
# add any missing states because we may not have used them all yet
# and cumsum would drop missing states from the result
model_states = adj_matrix.index.tolist()
missing_columns = set(model_states) - set(state_count_df.columns)
if missing_columns:
    for state in missing_columns:
        state_count_df[state] = 0
        # debugging is easier it we keep the state column-order canonical
        state_count_df = state_count_df[self.model_states]
# add the count aggregates to our running log
running_df = pd.concat([running_df, state_count_df], axis=1)
# turn the current row into its implied distribution
distribution_df = state_count_df.div(state_count_df.sum(axis=1), axis=0)
distributions = [row.to_frame() for _, row in distribution_df.iterrows()]

# ---> changes for distance are below
distances = []
for distribution in distributions:
    # make it look like the structure of the stationary distribution
    distribution.index.name = 'state'
    distribution.rename(columns={0:'probability'}, inplace=True)
    distance = distance_from_stationary(sample=distribution)
    distances.append(distance)
running_df['distance'] = distances
running_df['distribution'] = distributions

Now we will have that distance and its corresponding distribution available to us for visual comparison to the stationary distribution.

First, let’s look at a run with 200 seeds and 10 steps. We’ll keep the number of steps small this time in order to make it easier to examine details of the image.

Notice how at at the start (step 0), where all we have is the initial state, the two initial states that are closest to the stationary distribution are 'hotel' then 'airport'. This makes sense, as these are also the two states with the highest probabilities in the stationary distribution; 'hotel' is the highest probability and by distance measure it is the closest. At least JSD as the distance measure isn’t showing something entirely off-base right from the start.

As the run progresses, the median outcomes for each initial state converge, and by step 7 we’re seeing they don’t differ much from each other. The rest of the run going forward won’t likely be about specific initial states, but just the overall challenge of any state to result in data that closely approximates the stationary distribution.

Unfortunately we may be jumping the gun a bit at 7. Let’s look at seed sensitivity. We can plot the interquartile ranges on each step after the initial state:

The per-seed results are still jumping around quite a bit. This isn’t hugely surprising. The number of states is effectively a sample size, and when the sample size is something like 7 then it doesn’t take much to perturb the range of distances produced.

We can get an ever better look at this with initial states in separate plots.

So not only are the results variable per seed, the variability varies by initial_state. Clearly 7 steps is too soon to declare we’ve moved beyond bias.

We had our analytic estimate of 19 steps. Let’s take the analysis out to 30 steps so we can see 19 in a broader context.

The region around 19 steps looks quite stable. The variability is low, and low across all initial_state values. The rate of convergence is slowing down a lot, but that’s probably not a surprise. As a reminder from the previous article, the spectral gap being closer to 0 than 1 suggested convergence could be sluggish:

model.spectral_gap

Time to take a look at the actual histograms at step 19 and see what they tell us.

Not too bad. The estimated convergence seems to have relevance. Since this batch of data went out to 30 steps, we can compare to that.

Slightly better, but not game-changing. The important thing is that we aren’t seeing any meaningful bias based on what the initial state was, and as earlier plots showed the IQR variance based on the seed choice was narrowing as well.

Summary

• We wanted to generate data with this model and have a reasonable expectation that the data looked similar to what it was trained on. That is workable at 19 steps and would get progressively better over time.

• There would be no real need to worry about a better or worse choice of initial state or using data generated across multiple seeds, neither initial-state bias nor seed variance are problematic for chains at least 19 steps long.

• Our use of the Jensen-Shannon Distance appears safe. This isn’t really enough material to critique it versus other distance measures, but that wasn’t the goal. What we wanted was reasonably-similar histogram structure, which is what we have.

• The estimate of a mixing time (number of steps until viable convergence) lined up with the empirical results, which actually was a nice surprise. Going into the analysis I was skeptical that we would see decent histogram shape that early in the sequence.

References

• Jupyter notebook for this article.

• NetworkX API reference.

• Pandas API reference.

• PyDTMC PyPi.

• Häggström, Olle, “Stationary distributions,” in Finite Markov Chains and Algorithmic Applications. London Mathematical Society Student Texts, vol. 52, Cambridge University Press, 2002, pp. 28-38.

• Brémaud, Pierre, “Convergence Rates,” in Markov Chains: Gibbs Fields, Monte Carlo Simulation and Queues, Texts in Applied Mathematics, vol. 31, Springer Cham, 2020, pp. 289–329.

• “Total variation distance of probability measures,” in Wikipedia, accessed 2025-08-28, permanent link (revision) <https://en.wikipedia.org/w/index.php?title=Total_variation_distance_of_probability_measures&oldid=123456789>.

• “Markov chain mixing time,” in Wikipedia, accessed 2025-08-28, permanent link (revision) <https://en.wikipedia.org/w/index.php?title=Markov_chain_mixing_time&oldid=1233574639>.

The Experimentalist : Markov Chain Convergence in Python © 2025 by Reid M. Pinchback is licensed under CC BY-SA 4.0

The brief segue into finite graphs in the previous article has paved the way for developing a little intuition on Markov Chains and working with them in Python. We will want this material for future data engineering explorations.

The concept of a Markov Chain is easy enough:

• You have a collection of states.

• There are probabilities of transitioning from the current state to various future states.

• There is no memory behind those probabilities; if you know the current state then you know the probabilities of transitioning next to other states.

Prev: Finite Graphs and NetworkX | Next: Markov Chain Convergence in Python

We’re going to simplify the variety of Markov Chains we’ll be working with.

• We will only discuss chains with a finite number of states.

• Transitions from one state to the next happen in discrete time steps, not over the flow of continuous time. This lets us organize discussions in terms of time steps t0, t1, …

With those clarifications, what we’ll have are Discrete-Time Finite Markov Chains.

The use of the word ‘chain’ is intended to communicate that we can keep repeating this process of random state transitions over and over again. That assumes there are no states without any outgoing transitions, which are called absorbing states, which can terminate some randomly-generated walks.

The Graph Connection

Remembering our previous discussion on graphs, a Directed Finite Graph has:

• Nodes.

• Edges between pairs of nodes, and those edges are ordered from source to target.

• When an edge leaves a source node, that is an outedge.

• When an edge arrives (the arrow-head in a picture points at) a target node, that is an inedge.

• It is something useful to assign attributes like a numeric weight to edges.

We’ll add to this a little bit of simplified statistical terminology.

• A probability is a real number p where 0 ≤ p ≤ 1.

• A discrete probability distribution is a finite collection of probability values that add up to 1.

• If we have estimated or assigned probabilities to a collection of possible events, and those probabilities form a probability distribution, we refer to that collection of events as being stochastic.

And now we’re ready to characterize our Discrete-Time Finite Markov Chains (DTFMCs to save typing) in terms of Directed Finite Graphs (DFGs).

• States in a DTFMC are nodes in a DFG.

• If it is possible to transition between two states a and b in a DTFMC, then that is represented as the edge (a, b) in a DFG.

• The probability of such a transition from state a to state b in a DTFMC is represented as the weight of the edge (a, b).

• In an adjacency matrix for the DFG, each row corresponds to a source state and each column corresponds to a target state. All non-zero values will be the weights (probabilities of a possible state-to-state transition).

• For every node (state) in the DFG, its collection of weighted outedges (representing state transitions with non-zero probability) is stochastic.

That last two points tell us that when the DFG for a DTFMC is represented as an adjacency matrix, each row will either sum to:

• 0 if and only if that state has no outedges.

• 1 if and only if that state has at least 1 outedge.

This representation of a Markov Chain is sometimes referred to as row-stochastic or row-normalized. It would be possible to construct a matrix that instead was column-stochastic or column-normalized; in that case we would be seeing the probability distribution for how possible source states could transition to a target state. Row-stochastic is for understanding how a specific source leads to many potential targets, Column-stochastic is for understanding how many potential sources lead to a specific target. In this article we’ll be working with row-stochastic representations.

NetworkX Example

Consider the following taxi trip model as a Markov Chain:

• States are destinations.

• Edges are trips between destinations.

• Weight probabilities are the odds of going from one destination to another.

Our specific data will be

• From the airport, 70% of trips go to the hotel, 20% the hospital, 10% the mall.

• From the hospital, 60% of trips go to the hotel, 40% the airport.

• From the hotel, 40% of trips go to the mall, 40% the airport, 20% the hospital.

• From the mall, 70% of trips go to the hotel, 20% the airport, 10% the hospital.

As shown in the previous article, we can represent such a weighted DFG as an adjacency matrix:

adj_rows = dict(
    airport= [  0, 0.2, 0.7, 0.1],
    hospital=[0.4,   0, 0.6,   0],
    hotel=   [0.4, 0.2,   0, 0.4],
    mall=    [0.2, 0.1, 0.7,   0],
)
adj_matrix = pd.DataFrame.from_dict(adj_rows,
                                    orient='index',
                                    dtype=float,
                                    columns=list(adj_rows.keys()))
graph = nx.from_pandas_adjacency(adj_matrix, create_using=nx.DiGraph)

We can use NetworkX to describe the appearance of the graph. First we describe the nodes and node labels:

node_colors = dict(
    airport='purple',
    hospital='red',
    hotel='blue',
    mall='darkgreen',
)

for node_name, node_color in node_colors.items():
    nx.draw_networkx_nodes(graph,
                           pos=node_positions,
                           nodelist=[node_name],
                           node_color=node_color,
                           **node_params())
label_colors = {node_name: node_name for node_name in node_colors}
nx.draw_networkx_labels(graph,
                        pos=node_positions,
                        labels=label_colors,
                        **node_label_params())

Then we describe the edges and edge labels:

for node in graph:
    out_edges_view = typing.cast(nx.reportviews.OutEdgeView, graph.out_edges)
    nx.draw_networkx_edges(graph,
                           pos=node_positions,
                           edgelist=out_edges_view(node),
                           edge_color=node_colors[node],
                           **edge_params())
    for src, target, data in out_edges_view(node, data=True):
        edge_label = {(src, target): data['weight']}
        nx.draw_networkx_edge_labels(graph,
                                     pos=node_positions,
                                     edge_labels=edge_label,
                                     font_color=node_colors[node],
                                     **edge_label_params())

This renders as:

The outedges of each node are colored to match their source. The weights of all edges of the same color will add up to 1, reflecting their stochastic nature.

Now we have a visualization of a Markov Chain. From any state (node), in one discrete time step travel will happen on one of the correspondingly-colored outedges to a target node, with probability of that event as shown by the weight on the edge.

Counting DFG Walks

Going back for a moment to our graphs, we will introduce an additional concept.

• A walk is a sequence of connected edges in a graph.

• If the graph is a directed graph, edges only connect if each intermediate node functions first as a target for an edge, and then as a source for the next edge. Connections must respect directionality. If a graph has a directed edge (a, b), and another edge (c, b), you cannot directly connect those edges in a walk because they are going in opposite directions.

Given a particular DFG, between any two nodes, how many walks can we identify of length 2? Length 3? The answer is given by raising the unweighted adjacency matrix to the corresponding power. In other words if A is the adjacency matrix of a DFG, then the matrix product A * A computes the number of paths of length 2, and so on.

This can be illustrated with NetworkX and an edge list for weather transitions:

edge_rows = [
    dict(src='cloud', dest='cloud'),
    dict(src='cloud', dest='hail'),
    dict(src='cloud', dest='rain'),
    dict(src='cloud', dest='snow'),
    dict(src='cloud', dest='sun'),
    dict(src='hail', dest='cloud'),
    dict(src='hail', dest='hail'),
    dict(src='hail', dest='rain'),
    dict(src='rain', dest='cloud'),
    dict(src='rain', dest='hail'),
    dict(src='rain', dest='rain'),
    dict(src='rain', dest='snow'),
    dict(src='snow', dest='cloud'),
    dict(src='snow', dest='rain'),
    dict(src='snow', dest='snow'),
    dict(src='sun', dest='cloud'),
    dict(src='sun', dest='sun'),
]
edge_list_df = pd.DataFrame(edge_rows)
graph = nx.from_pandas_edgelist(edge_list_df,
                                source='src',
                                target='dest',
                                create_using=nx.DiGraph)
adj_matrix = nx.to_pandas_adjacency(graph, dtype=int)
adj_matrix

The last step converted the edge list to an adjacency matrix as a Pandas DataFrame:

This data restricts sun to not spontaneously transitioning to hail or snow or rain. We would need a walk of length at least 2 to have a chance to transition through cloudy weather first. We can accomplish that with matrix multiplication as a “dot” product:

adj_matrix @ adj_matrix

We can see, correctly, that there is only 1 walk of length 2 that starts with sun and ends with snow.

Also correct is the number of ways to start with sun and end with cloudy weather. There are 2: sun → sun → cloud and sun → cloud → cloud.

We don’t need to work with the adjacency matrix and figure out how many dot products to type in for higher powers. NetworkX provides a function to specify the desired power directly. It returns a dict of dicts, which we can make prettier as a Pandas DataFrame:

pd.DataFrame(nx.number_of_walks(graph, 2))

The results are the same as the calculation on the adjacency matrix.

The rest of the discussion will now return to exploring the taxi data.

Measuring DTFMC Walk Probabilities

In the DFG walk computations just shown, the weights in the adjacency matrix were all 1 to denote the directed edge. However, this same computation also works when the weights are other numbers, like probabilities. What is computed is the probability-weighted sum of those walks. This means:

• Compute the n’th power of a Markov Chain.

• We will get the probability of starting in a particular state a, and ending up in another state b. If there is no way to get from a to b then the value is 0.

• This is the probability to have that outcome after exactly n steps in the walk.

• The probability does not count the possibility of getting from a to b in fewer or greater than n steps.

Let’s try this for the taxi DTFMC defined earlier. The Pandas DataFrame for that adjacency matrix is:

We can compute the 3rd power of that graph. We’ll need to use the Pandas dot product for this, as NetworkX doesn’t understand weighting.

(adj_matrix @ adj_matrix) @ adj_matrix

As we can see, starting at the mall and returning again on the third trip has an 11.2% probability. Also note that no entries are zero. We’ll discuss the implications shortly.

Markov Chains and PyDTMC

So far we’ve just been exploring how to represent a Markov Chain as a graph with an associated matrix with per-state probability distributions. We can do more to analyze the characteristics of the model of taxi data that we’ve built.

import pydtmc as mc

model = mc.MarkovChain(adj_matrix)
print(model)

From this we can get a summary report about our model:

DISCRETE-TIME MARKOV CHAIN
 SIZE:           4
 RANK:           4
 CLASSES:        1
  > RECURRENT:   1
  > TRANSIENT:   0
 ERGODIC:        YES
  > APERIODIC:   YES
  > IRREDUCIBLE: YES
 ABSORBING:      NO
 MONOTONE:       NO
 REGULAR:        YES
 REVERSIBLE:     NO
 SYMMETRIC:      NO

The important characteristics for now are that the model is:

• Regular. This is a particularly strong property for a Markov Model to possess, and makes other operations much more approachable. It means that there is a probability distribution of states that all future state transitions will converge to.

• Ergodic. All the states are reachable from each other after a finite number of steps (irreducible), and the long-term distribution of states does not oscillate with periodicity (aperiodic). Above there was a place where it was pointed out that the adjacency matrix raised to the third power had all positive entries: that showed the matrix was recurrent.

• Recurrent: With a single recurrent class (meaning all states are in that class), we have that no matter which state we start in, we will return to it infinitely many times as we traverse more and more time steps.

Regular implies Ergodic, but the reverse is not always true.

Because we are dealing with Finite Markov Models, Recurrent implies Irreducible, and Irreducible implies Recurrent. These relationships would not hold for infinite models.

We can use PyDTMC to get that stationary distribution.

model.stationary_distributions

This is the probability distribution for visiting the states over time. The order of the states here is the same as the order of the rows in the original adjacency matrix, thus we spend the most time at the hotel: 0.4028777.

[array([0.25899281, 0.15107914, 0.4028777 , 0.18705036])]

We can obtain some insight into whether the model finds it easy or difficult to converge to that distribution. One way is by plotting the eigenvalues of the matrix:

mc.plot_eigenvalues(model)

From that we get:

Take notice of the two largest-magnitude eigenvalues. The wide red ring shows the distance between them, called the spectral gap. The closer they are, the more difficult it is for the future probabilities to generate walk that converge to the stationary distribution.

We can find out the spectral gap directly:

model.spectral_gap

• When the value is closer to 1, convergence is faster.

• When it is closer to 0, convergence is slower.

0.31078141211367716

In our case convergence is a little sluggish.

We can get further insight into convergence with Spectral Density.

model.density

This tells us how variable the data is:

0.9166666666666666

• A value near 1 suggests convergence is easier.

• Near 0 indicates convergence is harder, and it would still mean that no matter what the spectral gap may be suggesting.

While it may sound a little counter-intuitive, data that is more variable actually makes convergence easier. When it is less variable then you have some low-probability transitions in the matrix that can keep the model stuck instead of moving freely around all its states.

In the next article we’ll explore what that convergence behavior looks like.

References

• Jupyter notebook for this article.

• NetworkX API reference.

• Pandas API reference.

• PyDTMC PyPi.

The Experimentalist : Markov Chains with NetworkX and PyDTMC © 2025 by Reid M. Pinchback is licensed under CC BY-SA 4.0

Many areas of practical data analysis and systems reasoning are grounded in the idea of using a finite graph to represent a problem. Not only does this provide a way to structure data, but the structure itself may suit analysis by a variety of techniques.

We’re going to keep the context very straightforward in order to build intuition over a few articles. The dusty old math texts can stay on their shelves for this one.

Next: Markov Chains with NetworkX and PyDTMC

Finite Graph Terminology

Finite Graph: a finite collection of nodes and edges.

Node: an object in a graph that will have 0 or more edges connecting it to other nodes.

Edge: a line in the graph that connects two nodes. It is acceptable for the same node to be on both ends of a connection.

import networkx as nx
import matplotlib.pyplot as plt
import pandas as pd
import random
import typing

graph: nx.Graph = nx.Graph()
graph.add_nodes_from(['a', 'b', 'c', 'd', 'e'])
graph.add_edges_from([('a', 'b'), ('a', 'c'), ('b', 'd'),
                      ('c', 'c'), ('c', 'e'), ('e','b')])
node_positions: NodeLayout  = nx.circular_layout(graph)
nx.draw(graph, pos=node_positions, **draw_params())

Tuples like ('a', 'b') are the nodes at either end of that edge. As this is an undirected graph, ('b', 'a') would achieve the same.

The draw_params() function is a convenience for boilerplate reduction on setting rendering parameters across multiple examples.

def override_params(params: dict[str, typing.Any],
                    **kwargs: typing.Any) -> None:
    for key, value in kwargs.items():
        params[key] = value

def draw_params(**kwargs: typing.Any) -> dict[str, typing.Any]:
    params: dict[str, typing.Any] = dict(
        arrows=True,
        arrowsize=17,
        edge_color='darkgray',  # colors graph edges
        edgecolors='darkgray',  # colors border pixels of nodes
        font_color='black',
        font_size=15,
        font_weight='semibold',
        node_color='lightgray',
        node_size=525,
        width=2.5,
        with_labels=True,
    )
    override_params(params, **kwargs)
    return params

The complete code (available in a Jupyter notebook) renders:

Display of node labels is optional, which is a useful feature for large graphs. Just add with_labels=False to nx.draw() via the helper method draw_params():

nx.draw(graph, pos=node_positions, **draw_params(with_labels=False))

We can extend this idea a little further. We can give those edges a sense of direction.

Directed Finite Graph Terminology

Node: as defined previously.

Edge: a line in a graph that has a source node and a target node. When drawn, an arrow head points to the target node.

If the edges aren’t directed we call it an Undirected Finite Graph (or Undirected Graph for short), otherwise a Directed Finite Graph (again, Directed Graph for short).

Nodes in a directed graph now gain a bit more structure:

• Outgoing Edges: the directed edges connected to a node that have that node as a source. Also called “outedges” in some math texts.

• Incoming Edges: the directed edges connected to a node that have that node as a target. Also called “inedges” in some math texts.

Representing edges is no different, except now the order of the identifier pairs matters when it didn’t matter for undirected edges. The first member of a pair is the source, the second is the target. The only substantial difference is the change in the NetworkX class used when defining the graph. We change the class from nx.Graph to nx.DiGraph:

graph: nx.Graph = nx.DiGraph()
graph.add_nodes_from(['a', 'b', 'c', 'd', 'e'])
graph.add_edges_from([('a', 'b'), ('a', 'c'), ('b', 'd'),
                      ('c', 'c'), ('c', 'e'), ('e','b')])
node_positions: NodeLayout  = nx.circular_layout(graph)
nx.draw(graph, pos=node_positions, **draw_params())

With this change the rendering becomes:

Alternate Representations

It can be mathematically useful, particularly for directed graphs, to think of them as corresponding to a matrix. Each row of a matrix corresponds to a source node, and each column corresponds to a target node. This is referred to as an Adjacency Matrix.

adj_rows: dict[typing.Hashable, list[typing.Any]] = dict(
    a=[0, 1, 1, 0, 0],
    b=[0, 0, 0, 1, 0],
    c=[0, 0, 1, 0, 1],
    d=[0, 0, 0, 0, 0],
    e=[0, 1, 0, 0, 0]
)
adj_matrix = pd.DataFrame.from_dict(adj_rows,
                                    orient='index',
                                    dtype=int,
                                    columns=list(adj_rows.keys()))
graph = nx.from_pandas_adjacency(adj_matrix, create_using=nx.DiGraph)
node_positions: NodeLayout  = nx.circular_layout(graph)
nx.draw(graph, pos=node_positions, **draw_params())

We use the Pandas trick of providing a keyed dictionary for rows and specifying the orientation. Thinking in terms of a known source having various targets is more intuitive than what column-specified data would force on us: to think in terms of one target having multiple sources. It also allows the visual aid of the text for populating adj_rows to correspond to how we have been taught to conceive of matrices. All we need to do is inform Pandas what the column keys are, which in this case is the same as the row keys: columns=list(adj_rows.keys()).

The output is the same as before, which was the goal. This representation can be convenient when the existing understanding of a graph is organized in terms of outgoing edges, because all the outgoing edge connections for a node form the matrix row for that node.

The number 1 in the matrix indicates a directed edge from the source (the row index) to the target (the column index). Using 0 and 1 is a traditional starting point for adjacency matrices because it supports some useful linear algebra, but there is no constraint that only the number 1 be used to indicate a connection. Any non-zero value can represent, for example, weight or length.

The disadvantage of this representation, at least with NetworkX, is that the only information this can capture are node names, edge directions, and a single edge weight. Adjacency matrices for NetworkX do not support the idea of structured objects so we couldn’t use one to specify multiple per-edge attributes. For that we need Edge Lists.

edge_rows = [
    dict(src='a', dest='b'),
    dict(src='a', dest='c'),
    dict(src='b', dest='d'),
    dict(src='c', dest='c'),
    dict(src='c', dest='e'),
    dict(src='e', dest='b'),
]
edge_list_df: pd.DataFrame = pd.DataFrame(edge_rows)
graph = nx.from_pandas_edgelist(edge_list_df,
                                source='src',
                                target='dest',
                                create_using=nx.DiGraph)
node_positions: NodeLayout  = nx.circular_layout(graph)
nx.draw(graph, pos=node_positions, **draw_params())

The situation has changed a bit:

• Each edge must be specified as a dictionary with consistent keys indicating the source and target of the edge.

• The edge list is converted into a Pandas DataFrame, and the dictionary keys become the column names.

• We use nx.from_pandas_edgelist() to convert the DataFrame into a graph, which requires specifying the keys that were used for the source and target.

The final rendering though, at least for this data, is unchanged:

Now we can push this further to add more attributes to the edge data:

def random_color_channel() -> float:
    if not hasattr(random_color_channel, 'rand'):
        random_color_channel.rand = random.Random(42)
    return random_color_channel.rand.uniform(0.5, 1)

def random_rgb_color() -> tuple[float, float, float]:
    return random_color_channel(),
           random_color_channel(),
           random_color_channel()

edge_rows = [
    dict(src='a', dest='b', weight=1, color=random_rgb_color()),
    dict(src='a', dest='c', weight=3, color=random_rgb_color()),
    dict(src='b', dest='d', weight=2, color=random_rgb_color()),
    dict(src='c', dest='c', weight=1, color=random_rgb_color()),
    dict(src='c', dest='e', weight=2, color=random_rgb_color()),
    dict(src='e', dest='b', weight=1, color=random_rgb_color()),
]
edge_list_df: pd.DataFrame = pd.DataFrame(edge_rows)
graph = nx.from_pandas_edgelist(edge_list_df,
                                source='src',
                                target='dest',
                                edge_attr=['weight', 'color'],
                                create_using=nx.DiGraph)
edge_weights = nx.get_edge_attributes(graph, 'weight')
edge_colors = nx.get_edge_attributes(graph, 'color').values()
node_positions: NodeLayout  = nx.circular_layout(graph)
nx.draw(graph, pos=node_positions, **draw_params(edge_color=edge_colors))
nx.draw_networkx_edge_labels(graph, pos=node_positions,
                             **edge_label_params(edge_labels=edge_weights))

Now we’re adding some complexity:

• Edges have two attributes: weight and color.

• The color is an RGB triple where each channel is a float in the range [0, 1].

• When using nx.from_pandas_edgelist() to produce a graph we have to tell it about the per-edge attributes: edge_attr=['weight', 'color']

• NetworkX doesn’t do much with complex edges so we have to extract the data as edge_weights and edge_colors to use later as parameters.

• The nx.draw() function has limited smarts on some forms of rendering, so we include use of nx.draw_networkx_edge_labels() to render the weights as values on each edge.

• The way nx.draw() and nx.draw_networkx_edge_labels() share an understanding of node locations is via the node_positions layout object. In the previous examples it was only used once per rendering so its role was less apparent, but here we see it used to layer on rendering details. The object provides node coordinates in the rendering. I added a Python type alias to cover the kinds of data passed back by various layout algorithms:

NodeLayout: typing.TypeAlias = dict[typing.Hashable, list[float]]

Now we get a much richer rendering with per-edge weight labels and per-edge colors:

In future articles I expect to need to use various analytic techniques on graphs. Even without that, NetworkX can help with rendering images that support an explanation for how data is being transformed or analyzed.

References

• Jupyter notebook for this article.

• Matplotlib API reference.

• NetworkX API reference.

• Pandas API reference.

Edits

• 2025-08-22: Added terminology for inedges and outedges.

The Experimentalist : Finite Graphs and NetworkX © 2025 by Reid M. Pinchback is licensed under CC BY-SA 4.0

There are times when you need to chase down database client connections in PostgreSQL. Examples of the issues that arise are:

• Having too many idle connections, which can pointlessly throttle database performance due to the backend process resources allocated to each.

• Having too many connections in total relative to the semaphore array configuration of the database, resulting in new connections being denied.

• Holding transaction-scoped locks for too long or gaining them in a problematic order, which can create errors or deadlocks in other operations that need some of those same locks for work to proceed.

• Holding transactions open too long regardless of any locking issues, which can thwart maintenance operations like vacuums and backups.

• Burning through transaction ids at an unusual rate, which might trigger id wrap-around too fast for vacuum operations to keep up and thus cause the database to go into recovery mode.

• Connection activity may be unexpected or suspicious, in which case you may have a security incident to investigate.

Prev: PostgreSQL 17 on Vagrant and VirtualBox

The TCP/IP Story

When we set up server processes we often find ourselves having to configure one or more ports for access. We become very aware of server-side addressing via IP or DNS name plus a port. What is easy to forget is that the client end of the connection also has addressing information associated with it.

With each client-server interaction:

• There is a client process, and that process has a process id (PID).

• There is a server process, which also has a PID.

• Requests originate from the client and are sent as TCP/IP packets to the server.

• Responses originate from the server and are sent as TCP/IP packets to the client.

• The client finds the server via an IP address and TCP port. That port is the one we tend to be aware of as software engineers. When we configure a client we tell it the IP address — or indirectly, via a DNS name — and the port.

• The server also finds the client via an IP address and TCP port. We don’t configure the information into the server, so how does it know about the client?

The answer is in the IP and TCP headers for each packet.

• IP headers contain several fields, which include both source and destination IP addresses.

• TCP headers also contain several fields, including both source and destination TCP ports.

• The way a server knows where the client is located is by extracting those fields when a request is received, and using them when it generates a response.

• While the server port resulted from static configuration, client ports are often determined on-the-fly and sometimes referred to as “ephemeral” ports.

There is information missing from these packets: the PIDs. Nothing in the network traffic knows about processes. That association is handled by operating system support which the application processes make use of.

• The client process will use a socket and bind it to the client IP address and ephemeral TCP port.

• The server process will use a socket and bind it to the server IP address and statically-configured TCP port.

Ports and PostgreSQL

The picture changes when you introduce a PostgreSQL service into the equation. Since TCP/IP packets don’t know anything about processes, there is no reason to restrict ourselves to a single server process for the implementation.

PostgreSQL has several processes, but for this discussion we are concerned with two:

• The Postmaster is the process that listens to that statically-configured port known for establishing connections. The default is 5432.

• For each client connection the Postmaster will identify or launch a Backend process to do the work the client is requesting. The Backend is the process that will use the client IP address and ephemeral port for responding with the results.

There are more complicated processing scenarios in PostgreSQL but we’ll ignore them for today. What is shown here accounts for what is typical.

We’re simplifying the picture a bit here as clients and backends can continue this back-and-forth between requests and responses. The Backend is delegated to deal with both once the Postmaster finishes accepting the incoming client connection. The Postmaster will go back to listening for fresh connection attempts.

Note that 5432 is not an entirely arbitrary choice of number. There is an official IANA registration for the service name and port. While you can configure PostgreSQL to use other ports, by sticking with 5432 you will find some network-aware Linux tools will automatically know that “5432” and “postgresql” are interchangeable.

Gathering Client Connection Data

All of this connection and request/response activity isn’t just hidden away in the memory state of the various processes. We can examine it. Some of that examination can happen within PostgreSQL itself, while some needs to be done with O/S tools.

We’ll explore this via the Vagrant/VirtualBox VM from the previous article. Refer to that if you haven’t already spun up your own PostgreSQL sandbox.

# cd to the project directory with the Vagrantfile before proceeding
vagrant up --provision
vagrant ssh

Now we can use a psql session to do some looking around. We’ll use the vagrant database and username as we previously created that as a superuser. As a memory-jogger we just used the trivial password “vagrant” for that sandbox. We’ll get into the details of database authentication and security in future articles.

psql --host=localhost --dbname=vagrant --username=vagrant

You don’t have to specify the dbname as it is the one associated with the username, but it’s worth seeing an example of how to when you need to.

The ability to examine client connection activity is handled in PostgreSQL via the Cumulative Statistics System. Under-the-hood PostgreSQL uses internal tables to gather activity data which you then examine via pre-defined “system views”. We’re going to put one of them to work: pg_stat_activity.

Our psql session is the only client we currently have connected (as shown by the backend_type being '“client backend”), and we can see it there in the response. As we are running on the same host as the database service and connected via “localhost”, our client IP address is correctly 127.0.0.1 as shown for client_addr. The client-side ephemeral port for the connection is 41592 as shown for client_port.

Notice the “pid” column. What is that 10614 process id for? Let’s look at the PostgreSQL processes just like we did in the previous article:

This has one more row than was shown in the previous article. That bottom row is the corresponding backend for our psql session. The “pid” shown in psql thus was for the backend process, not the client process. The ps output even shows the client IP address and ephemeral port that particular backend is handling.

Also compare the first entry of the ps results to our query response. That process doesn’t show in psql at all! That’s because it is the parent process with the Postmaster functionality. It doesn’t connect to the database, it is the thing that knows connections are something to be managed along with the processing resources assigned to them. As it doesn’t connect to directly perform database work, pg_stat_activity doesn’t know about it.

We only have that single psql session so it’s trivial to identify the client. We’re going to increase the difficulty by launching two more clients. One will be in another psql session on the same VM, and the other will be a psql session started from the underlying host that the VM is running on. As a reminder from the last article, we configured pg_hba.conf to allow connections via the mask 10.0.2.0/24 and here you are seeing why: the VM’s virtual network mapped the host to an address in that range.

Now we have three psql “client backend” sessions and no real way to tell between them. How do we figure out which client matches up with which backend? All we have are the backend process ids, but that tells us nothing about each client.

Remember earlier we noted that sockets are what associate processes with IP addresses and ports? That’s how we figure can this out. In the previous article one of the provisioning steps installed the iproute2 package, which provides us with ss.

This shows us both the client and server connections that are on the VM.

• You can ignore the “sshd” rows, those are for the terminal sessions I created.

• sudo ss -tunp ensures we see the sockets and corresponding process data for all users.

• ss -tunp (i.e. no use of sudo, running as vagrant); would have only shown “Process” column details for the client process sockets of the current user.

• sudo -U postgres ss -tunp would have only shown “Process” column details for the backend process sockets. The backend processes run as the postgres user.

• None of these shows the client details for the psql session launched from outside the VM. The only way to know client details is by doing this form of examination on the device corresponding to the client IP address.

• The two entries with “psql” are the clients running on the VM. Each shows the “Local Address:Port” and corresponds to the “client_addr,client_port” columns shown earlier in the pg_stat_activity response. The PID showing is for the client process, not the backend.

• The three entries with “postgres” are the backends running on the VM. Each has the PID that corresponds to the “pid” column shown earlier in the pg_stat_activity response. The “Peer Address:Port” corresponds to the client IP and port that backend will respond to.

As a quick aside, the “postgres” backend for the remote (host) psql client looks a little different than the other two cases. That psql client is not connecting via localhost, it is connecting via an IP address assigned by the VM’s virtual network. We can confirm that “10.0.2.15” is indeed the IP address assigned to the VM:

We are almost there.

• Get the “Local Address:Port” on the “psql” rows.

• Get the “Peer Address:Port” on the “postgres” rows.

• Match them up. In database terms, do a join on (IP, Port) across both subsets.

• Get the “pid” value from the “Process” columns. There will be two per match-up, one for the client and one for the backend.

|-------------|----------------|-----------------|------------|
| Backend PID |   Backend Addr |     Client Addr | Client PID |
|-------------|----------------|-----------------|------------|
|       10614 | 127.0.0.1:5432 | 127.0.0.1:41592 |      10612 |
|       10730 | 127.0.0.1:5432 | 127.0.0.1:47160 |      10737 |
|       10741 | 10.0.2.15:5432 |  10.0.2.2:54092 |          ? |
|-------------|----------------|-----------------|------------|

The final case is a little different because I’m running VirtualBox on Windows. For that we need to run netstat in a terminal window with privilege escalation:

That gives us similar information for the third client, including how the backend connection is being proxied by VirtualBox. The PID for the proxy won’t correspond to anything within the VM and can be ignored.

We now have a client-process-to-backend-process correspondence! Completing the table required using either netstat or ss and having access to the appropriate client host, as identified by “client_addr” in pg_stat_activity. Discovering the client PIDs allows us to investigate those processes on those hosts. We weren’t even constrained to only searching on Linux.

Could you automate all this? Yes, assuming the client hosts are accessible to you. That’s a discussion for another day.

The Experimentalist : Client Connectivity with PostgreSQL © 2025 by Reid M. Pinchback is licensed under CC BY-SA 4.0

In the article series on an exploratory lab I’ll eventually tackle provisioning of container and virtualization-based sandboxes. To tease at the benefits, we’re going to use a Vagrant-VirtualBox setup to install and poke around a running database.

If you aren’t already using them, you can download and install in order:

• VirtualBox → https://www.virtualbox.org/wiki/Downloads

• Vagrant → https://developer.hashicorp.com/vagrant/install

Next: Client Connectivity with PostgreSQL

Project files for the article are available from the newsletter’s GitLab repository:

• Vagrantfile

• provisions/postgresql.sh

Just create a project that maintains the directory structure. Once you’ve done that and installed both VirtualBox and Vagrant:

• cd to the project directory

• vagrant up

This will get the ball rolling. If the install will go slowly, it’ll usually be right after generating the SSH key. That tends to happen if there is some difficulty in mapping hardware functionality that the O/S expects to what VirtualBox is able to provide. I’ve found the Ubuntu 22.04 Bento box to work pretty smoothly. It may take 10-20 minutes to download an O/S image, build a VM from it, and install PostgreSQL, and the install will conclude with:

Any future restarts of the box will be much faster. At this point you have:

• A running Ubuntu 22.04 VM.

• A running PostgreSQL 17 service.

• An Ubuntu login account named “vagrant” you can ssh to.

• An PostgreSQL user named “vagrant” with the password “vagrant”.

• The ability to connect to the database as that user with either a client running within the VM, or from a client on the underlying host operating system.

Let’s take a look at how that came about. The driver of the bus is the Vagrantfile:

This is the file from the GitLab repository but with comments stripped out. Let’s walk through the key parts.

Specify the O/S image you want for the VM:

config.vm.box = "bento/ubuntu-22.04"
config.vm.box_version = "202508.03.0"

PostgreSQL by default listens on port 5432. We want to make it possible for a database client running on the host to connect to the database service within the VM on that same port number:

config.vm.network "forwarded_port", guest: 5432, host: 5432, \
  host_ip: "127.0.0.1"

We specify that the provider is VirtualBox, but if you happen to already use Hyper-V or VMware, this is where you would make the change to use them instead:

config.vm.provider "virtualbox" do |vb|

Finally we have a script to run as root for provisioning the VM once it is ready. Ours sets up PostgreSQL and does other assorted housekeeping:

config.vm.provision "shell", name: "postgresql", \
  path: "provisions/postgresql.sh", privileged: true

All of the real magic happens in the provisioning script. First we set up the firewall and install some utilities that may come in handy later:

Then we install the APT repository for PostgreSQL, and install the service:

The next step is a bit more packed and needs explaining.

We want to configure the database to have an initial user and database schema to work in. Vagrant allows us to re-run provisioning scripts. Those scripts should be idempotent so they don’t fail they next time they are run. A zero-byte file named .configured is created as a marker to note we’ve already been here before.

The work of the provisioning on the first pass consists of:

• Change postgresql.conf to listen for connections coming in from any host. This is done so that it doesn’t just respond to VM clients, it would also respond to the any client the Vagrant configuration allows access to.

• Create a user named vagrant, which for simplicity here I’ve made a superuser just like the default user postgres already is.

• Provide the vagrant user with a password to allow login. Password-based authentication is not the only option, but we’re going for simple.

• Create a database for the new user and give it the same name as that user.

• Change pg_hba.conf to allow the new user to have access. We specify both the loopback address to allow login from within the VM, and an address mask suited to any connections over eth0 with IP addresses assigned to the VM or host.

• Restart the database service once everything is ready. If this was a production instance I would dig into the docs to remind me which postgresql.conf features require a restart versus a config reload. For here this works fine, and I want a chance to confirm the systemd unit for the service in any case.

Some of the steps are wrapped in a heredoc to run as the postgres user. We are using nested heredocs: the outer level to process steps as postgres, and the inner level to specify content to append to the pg_hba.conf file. The entire provisioning script runs as a privileged user by Vagrant because we said so in the Vagrantfile, which for us means as root. The PostgreSQL service runs as postgres.

Finally, the provisioning script cleans up any dangling APT state. If anything installed flagged a need to reboot the host (in this case, the VM) to complete an installation, then it gets handled now.

We now have the ability to ssh into the running VM. We use vagrant ssh for that, because it knows about the key pair that Vagrant generated when the VM was created.

As a final step, we’ll confirm that the database service processes exist:

In the next article we’ll start exploring the service in more detail.

The Experimentalist : PostgreSQL 17 on Vagrant and VirtualBox © 2025 by Reid M. Pinchback is licensed under CC BY-SA 4.0