Assignment 1
Downloads
Technique
Three random sets of coefficients are created. These are run against each other, themselves, and the coefficients from the previous assignment. Each time the evaluation function runs with some coefficients, the coeffcients are changed according the the least mean squares training algorithm, with the result of the old evaluation function used as the training set.
It should be noted that the evaluation function was modified. Previously, the evaluation function gave the coefficients multiplied by the features with a final game state. Now, a final game state yields 1 for a win, 0 for a tie, and -1 for a loss. This rendered the first two features, win and loss, irrelevant, so those are now always zero. This means that the first two coefficients don't actually do anything.
Results
Using this technique, the coefficients all converged towards those found in the last assignment, with the exception of the win and loss coefficients, which were rendered meaningless after changing the evaluation function.
| Features | Old coefficients | New coefficients |
| Win | 0.2502943943301069 | 0.32386133725050104 |
| Loss | -0.4952316649483701 | 0.5490470895311659 |
| Own Even Threats | 0.3932539700819625 | 0.3932539698522742 |
| Other's Even Threats | -0.2742452616759889 | -0.27424526114286796 |
| Own Odd Threats | 0.4746881137884282 | 0.4746881136468789 |
| Other's Odd Threats | 0.2091091127191147 | 0.2091091126568151 |
Given the similarities between the two sets of coefficients, it would be reasonable to expect identical play between the two. However, this is not these case. There are two cases: when the new coefficients are red and the old coefficients are black, or vice versa. Black wins in either case, but differently. In both cases, red ignores that black is about to win and tries to make a 3-in-a-row.
Comparison of old coefficients and new coefficients
Here are two runs of the game, with commentary. I attempted to record this, but 7 moves occurred in a single frame. As a quick and dirty hack, I used the text UI, which will also allow me to add commentary easily.
Our first game is with the old coefficients as red and the new as black.
0123456
It is red's turn.
R
0123456
Red starts in the center, which is almost always a good move.
It is black's turn.
B
R
0123456
It is red's turn.
R
B
R
0123456
Both keep placing in the middle, due to the fact that, all else being equal, they make moves in the middle by default, if all utilities are the same.
It is black's turn.
R
B
RB
0123456
Black builds sideways. Perhaps to build a line outward?
It is red's turn.
R
R
B
RB
0123456
Red is reaching up for a line.
It is black's turn.
B
R
R
B
RB
0123456
However, red is blocked off.
It is red's turn.
B
R
R
BR
RB
0123456
It looks like red is trying to build a diagonal now.
It is black's turn.
B
R
R
BR
RBB
0123456
But black sees right through that.
It is red's turn.
B
R
RR
BR
RBB
0123456
This move opens up quite a few options for red, horizontally, vertically, or diagonally.
It is black's turn.
B
R
RR
BR
RBBB
0123456
I'm really not sure what black is doing here. Maybe it just likes rows of three?
It is red's turn. R B R RR BR RBBB 0123456
Red seems to be idling here.
It is black's turn. R B RB RR BR RBBB 0123456
Black sets up a far future diagonal, but is probably really trying to block red from a vertical win.
It is red's turn. R B RB RR BRR RBBB 0123456
Red also sets up a far future diagonal, but one that's a bit more plausible. It also could help with a horizontal win.
It is black's turn. R B RB RRB BRR RBBB 0123456
Black sets up for an immediate diagonal win, but could be blocked easily, which is probably why red allowed it.
It is red's turn. R B RB RRB BRRR RBBB 0123456
As expected, red blocks.
It is black's turn. R BB RB RRB BRRR RBBB 0123456
Black appears to be trying to setup a few distant options.
It is red's turn. RR BB RB RRB BRRR RBBB 0123456
I suspect red is too, but it might just be idling.
It is black's turn. RR BB RB RRB BRRR BRBBB 0123456
We finally have play on the left side of the board. I don't know what black is trying here.
It is red's turn. RR BB RB RRB RBRRR BRBBB 0123456
But red looks to be setting up for a horizontal three-in-a-row, or possibly a diagonal.
It is black's turn. RR BB RB BRRB RBRRR BRBBB 0123456
However, black blocks both, while also building its own three-in-a-row.
It is red's turn. RR BB RBR BRRB RBRRR BRBBB 0123456
Not sure what red's doing here.
It is black's turn. RR BBB RBR BRRB RBRRR BRBBB 0123456
But black is setting up a three-in-a-row, so that seems especially stupid of red.
It is red's turn. RR BBB RBR BRRB RBRRR R BRBBB 0123456
Red looks like it's just trying to gain a hold of the leftmost side.
It is black's turn. RRB BBB RBR BRRB RBRRR R BRBBB 0123456
While black looks to be blocking a possible three-in-a-row.
It is red's turn. RRB BBB RBR BRRBR RBRRR R BRBBB 0123456
Still not sure what red's doing. Possibly idling at this point.
It is black's turn. RRB BBB RBR BRRBR RBRRR RBBRBBB 0123456
Black finally is pushing further left.
It is red's turn. RRB BBB RRBR BRRBR RBRRR RBBRBBB 0123456
Red builds a three-in-a-row here, but sets up black for an immediate victory. It looks like red could have held out longer by moving to column 0 or 1, which do not appear to be dangerous. It is possible that red concludes that it cannot win and moves for the center, but I suspect that weight from building this three-in-a-row actually was higher than the cost of losing.
It is black's turn. RRB BBBB RRBR BRRBR RBRRR RBBRBBB 0123456
And black takes the win.
It is red's turn. Black wins!
Let's do this again, but with the players switched. So, red is played by the new coefficients and black is played by the old ones.
0123456
It is red's turn.
R
0123456
A standard start.
It is black's turn.
B
R
0123456
It is red's turn.
R
B
R
0123456
It is black's turn.
B
R
B
R
0123456
It is red's turn.
B
R
B
RR
0123456
This is largely a repeat of last time, except red broke of the central stack first.
It is black's turn.
B
R
B
BRR
0123456
Black plays opposite, gaining an advantage on the left to counter red's advantage on the right.
It is red's turn.
B
R
RB
BRR
0123456
But red quickly builds 2 two-in-a-rows.
It is black's turn.
B
B
R
RB
BRR
0123456
Looks like black is going vertical, perhaps for a win in the far future?
It is red's turn. R B B R RB BRR 0123456
Red grabs that top spot.
It is black's turn. R B B R RBB BRR 0123456
So, black builds a two-in-a-row.
It is red's turn. R B B R RBB RBRR 0123456
Red builds a nice diagonal three-in-a-row here.
It is black's turn. R B B BR RBB RBRR 0123456
Black builds 2 two-in-a-rows, similarly to how red did earlier.
It is red's turn. R B B BR RBB RBRR R 0123456
Red pushes right, causing an imminent win, which should force black.
It is black's turn. R B B BR RBB RBRRBR 0123456
As it does.
It is red's turn. R B B BR RBB RRBRRBR 0123456
Red looks to be establishing a firmer position on the left here.
It is black's turn. R B B BR RBB B RRBRRBR 0123456
In a mirror of red's recent threat, black is forcing red to move to column 5.
It is red's turn. R B B BR RBBRB RRBRRBR 0123456
Which it does. It also looks like it helps red a little.
It is black's turn. R B B BR B RBBRB RRBRRBR 0123456
Black blocks something that I noticed as a possible horizontal win, which could have forced black to allow a red diagonal win by forcing black to play in 4, followed by red playing 4 for the win.
It is red's turn. R B B R BR B RBBRB RRBRRBR 0123456
Not sure what red's doing here. Possibly setting up a diagonal in the far future, or just avoiding bad moves elsewhere.
It is black's turn. R B BB R BR B RBBRB RRBRRBR 0123456
Here, black forms 3 two-in-a-rows, which is nice.
It is red's turn. R RB BB R BR B RBBRB RRBRRBR 0123456
Red blocks off one of them and forms a two-in-a-row.
It is black's turn. R RB BB R BR B BRBBRB RRBRRBR 0123456
This doesn't keep black from forming a three-in-a-row here.
It is red's turn. R RB BB R RBR B BRBBRB RRBRRBR 0123456
Red simulataneously forms a three-in-a-row, while giving black 2 three-in-a-rows. It's not a terrible move, but it's not great.
It is black's turn. R RB BBB R RBR B BRBBRB RRBRRBR 0123456
Unsurprisingly, black takes them, while blocking a red three-in-a-row.
It is red's turn. RR RB BBB R RBR B BRBBRB RRBRRBR 0123456
Red builds 2 two-in-a-rows, though only one of them could be useful.
It is black's turn. RR RB B BBB R RBR B BRBBRB RRBRRBR 0123456
Black appears to be setting up a broken two-in-a-row, but looks more to be idling.
It is red's turn. RR R RB B BBB R RBR B BRBBRB RRBRRBR 0123456
Red sets up a broken three-in-a-row here. I suspect neither like moving elsewhere.
It is black's turn. RR R BRB B BBB R RBR B BRBBRB RRBRRBR 0123456
For example, this gives black a two-in-a-row, but will give red a three-in-a-row immediately.
It is red's turn. RR R BRB B BBB R RBRRB BRBBRB RRBRRBR 0123456
However, it looks like red makes a severely silly move here, setting black up for an immediate win. It's quite possible that red cannot win, but I see three ways red could win, though I haven't played them out, so they could be impossible. That's the only plausible reason I see for taking this move. I think that, perhaps, evaluation functions should favor losing later, rather than sooner, as humans make mistakes that may not be considered by minimax.
It is black's turn. RR R BRB B BBBBR RBRRB BRBBRB RRBRRBR 0123456
Finally, black takes the win.
It is red's turn. Black wins!