Repository navigation
Add AS algorithm. - #65
Conversation
Codecov Report
@@ Coverage Diff @@
## master #65 +/- ##
==========================================
+ Coverage 94.06% 94.58% +0.52%
==========================================
Files 5 5
Lines 337 388 +51
==========================================
+ Hits 317 367 +50
- Misses 20 21 +1
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Wow! This is great @shizejin -- When I wrote the original outerapproximation code, I had meant to do this algorithm as well but never got around to it. I am very excited that you have implemented it.
I've downloaded your notebook and can reproduce all of your graphs.
This is a first glance through the code.
Overall, I think things look very good. I'd love to see if we can make this code as fast as their implementation (or faster!).
Thanks @shizejin .
|
|
||
| for iter = 1:max_iter | ||
|
|
||
| v_new = Vector{T}(0) # to store new vertices |
There was a problem hiding this comment.
If you don't know how many vertices you will need beforehand, I recommend using sizehint!(v_new, XYZ) where XYZ is maximum number of vertices you might have. If I remember correctly, they are able to pin down maximum number of vertices possible for finite action games (It might have been 4 * nactions?).
There was a problem hiding this comment.
Yes, the maximum number of vertices is 4 * num_actionprofiles. I will use sizehint!(v_new, 8 * num_actionprofiles) then. Thanks!
| end | ||
|
|
||
| # create VRepresentation and Polyhedron | ||
| V = SimpleVRepresentation(v_old) |
There was a problem hiding this comment.
These lines seem to be creating some sort of type instability -- I couldn't quite work out where it was coming from, but given that you call these functions again later, you might worry that the performance losses are coming from this instability (I'm not totally sure though)
julia> @code_warntype AS(rpd, 1000, plib, 1e-5, [0.0, 0.0])
Variables:
#self#::Games.#AS
rpd::Games.RepeatedGame{2,Float64}
max_iter::Int64
plib::Polyhedra.SimplePolyhedraLibrary{Float64}
tol::Float64
u@_6::Array{Float64,1}
i@_7::Any
#temp#@_8::Any
a1::Int64
payoff1::Float64
payoff2::Float64
IC1::Float64
IC2::Float64
p_IC::Polyhedra.SimplePolyhedron{_,Float64} where _
p_inter::Any
#temp#@_16::Int64
a2::Int64
#temp#@_18::Int64
iter@_19::Int64
v_new::Any
u_::Array{Float64,2}
#temp#@_22::Int64
v_old::Array{Float64,2}
H::Polyhedra.SimpleHRepresentation{_,Float64} where _
V::Polyhedra.SimpleVRepresentation{_,_} where _ where _
p::Polyhedra.SimplePolyhedron{_,Float64} where _
...
There was a problem hiding this comment.
This part is also what I am really concerned about. What I am trying to do here is to use a Matrix v_new to create VRepresentation without any redundant points (because the computation of intersection of two polyhedra is really slow if there are many redundant points).
I am not sure why Polyhedra.jl do not implement removevredundancy!(p::VRep), and I have to call the internal function removevredundancy(vrep::VRep, hrep::HRep; strongly=true) to remove the redundant points, which I don't know if it's efficient or not. I think we need to look into Polyhedra.jl deeply to see if I can improve this (very important) part.
| end | ||
|
|
||
| # find out the intersections of polyhedron and IC boundaries | ||
| p_IC = polyhedron(SimpleHRepresentation(-eye(2), -[IC1, IC2]), plib) |
There was a problem hiding this comment.
Could you precompute these?
Something like
p_IC = [polyhedron(SimpleHRepresentation(-eye(2), [-IC1[i], -IC2[i]]), plib) for i in naction]
where IC1 and IC2 were appropriately defined vectors? The IC constraints don't change, right? This might help with type stability since they'll be known before the for loop? I'm uncertain about this though
There was a problem hiding this comment.
IC constraints will change when u is updated. But yeah, I think there is no need to update p_IC in every iteration. I will move this into here so that p_IC is updated only if u changes. This will save a lot of time! Thanks.
|
|
||
| # step 2 | ||
| # update u | ||
| u_ = [minimum(v_new[:, 1]) minimum(v_new[:, 2])] |
There was a problem hiding this comment.
Given that u is always one dimensional, I'd put a comma here to make u_ one dimensional as well.
Below when you write u = u_ you are changing u between a 1 dimensional array and 2 dimensional array (this might also be a source of instability). See below for example
julia> u = [rand(), rand()]
2-element Array{Float64,1}:
0.0212796
0.144139
julia> u_ = [rand() rand()]
1×2 Array{Float64,2}:
0.8921 0.408161
julia> if all(u_ .> u)
u = u_
end
1×2 Array{Float64,2}:
0.8921 0.408161
julia> u
1×2 Array{Float64,2}:
0.8921 0.408161
There was a problem hiding this comment.
You are right. At the beginning I set both u and u_ to be two dimensional, but then I change u to one dimensional, while forget to change u_ as well. Thanks for finding this out!
|
Thanks for your review @cc7768! It is very helpful. There are two things in my mind that might improve the efficiency:
|
7d4b2ee to
4fbf004
Compare
|
@shizejin What is the status of this PR? |
|
@oyamad Currently the performance is still not as desirable. But it seems that there has been many updates in |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
# Conflicts: # src/Games.jl
|
To update the Polyhedra.jl functions, I just almost blindly followed the fixes proposed by ChatGPT 5 (a592647). But tests pass now anyway (the errors with "CI-nightly / Julia nightly" are not caused by this PR). |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> (+3 squashed commits) Squashed commits: [4a5de9e] Fix Copilot suggested changes [4313e60] Update src/repeated_game.jl Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> [3d8e032] Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Following suggestions from ChatGPT 5
3b72233 to
0a8f623
Compare
|
@shizejin |
Tests fail with this commit
|
|
||
| for iter = 1:maxiter | ||
|
|
||
| v_new = T[] # to store new vertices |
There was a problem hiding this comment.
If T == Int, then this will be a vector of Ints, which will lead to InexactError.
EDIT: Maybe force |
There was a problem hiding this comment.
Pull Request Overview
This PR implements the AS algorithm proposed by Abreu and Sannikov (2014) for computing the set of payoff pairs of all pure-strategy subgame-perfect equilibria with public randomization in repeated two-player games. The algorithm provides significant efficiency gains over the existing JYC algorithm by better understanding how extreme points of the equilibrium payoffs set are generated.
Key changes include:
- Implementation of the AS algorithm with polyhedra-based computations
- Addition of comprehensive test coverage for the new algorithm
- Export of the new AS function in the main module
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| src/repeated_game.jl | Adds the main AS algorithm implementation along with helper functions for payoff point extraction and deviation gain calculations |
| test/test_repeated_game.jl | Adds comprehensive test suite for the AS algorithm covering different numeric types (Float64, Int, Rational) |
| src/GameTheory.jl | Exports the new AS function to make it available to users |
| if size(v_dedup) == size(v_old) | ||
| # then check the euclidean distance | ||
| if norm(v_dedup-v_old) < tol | ||
| println("converged in $(iter) iterations") |
There was a problem hiding this comment.
Using println for output in a library function is not appropriate. Consider using the @info macro from the Logging module instead, or make the verbose output optional through a parameter.
| println("converged in $(iter) iterations") | |
| @info "converged in $(iter) iterations" |
| end | ||
|
|
||
| # find out the intersections of polyhedron and IC boundaries | ||
| p_IC = polyhedron(hrep(-Matrix{Float64}(I, 2, 2), -[IC1, IC2]), plib) |
There was a problem hiding this comment.
The Matrix{Float64} type is hardcoded, which will cause type mismatches when T is not Float64 (e.g., when using Rational or Int types). This should use Matrix{T} instead to match the function's generic type parameter.
| p_IC = polyhedron(hrep(-Matrix{Float64}(I, 2, 2), -[IC1, IC2]), plib) | |
| p_IC = polyhedron(hrep(-Matrix{T}(I, 2, 2), -[IC1, IC2]), plib) |
| # Use deduplicated vertices for convergence check | ||
| v_dedup = MixedMatVRep(vrep(p)).V | ||
| # first check if the numbers of vertices are the same | ||
| if size(v_dedup) == size(v_old) |
There was a problem hiding this comment.
Comparing matrix sizes with == may not work as expected in all cases. Use size(v_dedup, 1) == size(v_old, 1) && size(v_dedup, 2) == size(v_old, 2) for more reliable size comparison.
| if size(v_dedup) == size(v_old) | |
| if size(v_dedup, 1) == size(v_old, 1) && size(v_dedup, 2) == size(v_old, 2) |
| # first check if the numbers of vertices are the same | ||
| if size(v_dedup) == size(v_old) | ||
| # then check the euclidean distance | ||
| if norm(v_dedup-v_old) < tol |
There was a problem hiding this comment.
[nitpick] Computing the norm of the difference between two matrices can be expensive for large matrices. Consider using a more efficient convergence criterion, such as checking the maximum absolute difference between corresponding elements.
| if norm(v_dedup-v_old) < tol | |
| if maximum(abs.(v_dedup .- v_old)) < tol |
|
Let me merge this, and later let's make it type flexible. |
…ew comments Co-authored-by: oyamad <7353083+oyamad@users.noreply.github.com>
…boundary intersection and incremental redundancy removal Co-authored-by: oyamad <7353083+oyamad@users.noreply.github.com>
… analysis files Co-authored-by: oyamad <7353083+oyamad@users.noreply.github.com>
…191) * Initial plan * Implement flexible AS type support: extend RepeatedGame struct, add _coefficient_type function, update AS function signature, add uniquetolrows utility Co-authored-by: oyamad <7353083+oyamad@users.noreply.github.com> * Add comprehensive tests for flexible AS types and address PR #65 review comments Co-authored-by: oyamad <7353083+oyamad@users.noreply.github.com> * Finalize flexible AS implementation: handle Int payoffs with rational delta, comprehensive type safety Co-authored-by: oyamad <7353083+oyamad@users.noreply.github.com> * Fix missing end statement for for loop in AS function Co-authored-by: oyamad <7353083+oyamad@users.noreply.github.com> * Fix AS algorithm tests to handle variable vertex count in rational case Co-authored-by: oyamad <7353083+oyamad@users.noreply.github.com> * Implement PR #65 efficiency optimizations: add optimization-based IC boundary intersection and incremental redundancy removal Co-authored-by: oyamad <7353083+oyamad@users.noreply.github.com> * Add comprehensive performance analysis for AS algorithm optimizations Co-authored-by: oyamad <7353083+oyamad@users.noreply.github.com> * Add comprehensive performance analysis of AS algorithm optimizations Co-authored-by: oyamad <7353083+oyamad@users.noreply.github.com> * Revert last 3 commits: remove PR #65 optimization implementations and analysis files Co-authored-by: oyamad <7353083+oyamad@users.noreply.github.com> * Address PR review comments: optimize performance and fix formatting - Replace array operations with scalar comparisons for performance - Preallocate payoffs vector to avoid allocations - Precompute A_IC matrix outside main loop - Fix matrix initialization using npoints(vr) instead of length(pts) - Fix indentation in u array initialization - Sort rows before convergence check to handle variable vertex order - Add blank line between function end and docstring - Update documentation with exact arithmetic information - Remove internal function references from public documentation Co-authored-by: oyamad <7353083+oyamad@users.noreply.github.com> * Fix AS algorithm convergence check: remove sorting that caused premature convergence Co-authored-by: oyamad <7353083+oyamad@users.noreply.github.com> * Revert "Fix AS algorithm convergence check: remove sorting that caused premature convergence" This reverts commit 8d59120. * FIX: Force `u` no smaller than minimum payoffs * Add Examples to `AS` * Fix doc [ci skip] * Revise doc [ci skip] --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: oyamad <7353083+oyamad@users.noreply.github.com> Co-authored-by: Daisuke Oyama <oyama@e.u-tokyo.ac.jp>
An implementation of AS algorithm proposed by Abreu and Sannikov 2014. This algorithm is for computing the set V* of payoff pairs of all pure-strategy subgame-perfect equilibria with public randomization. Comparing to the JYC algorithm (which has been nicely implemented by @cc7768), this algorithm has significant efficiency gains, which arise from a better understanding of the manner in which extreme points of the equilibrium payoffs set are generated (in other word, this algorithm drops a lot of points that will not be included in V*, which will be kept by JYC algorithm during the iteration process).
Here demonstrates the usage of this algorithm with 3 examples. The first two are examples from the paper, and the third is a simple Prisoner Dilemma game that used in the notebook of JYC algorithm. I did comparison between JYC and AS algorithm using the third example.
I use
Polyhedra.jlto do computation of polyhedra. Because this algorithm computes polyhedra intensively, the choice ofPolyhedraLibrarywill influence the speed very much. As the result shows, in the simple Prisoner Dilemma game, the AS algorithm is approximately twice faster than JYC algorithm if we useSimplePolyhedraLibrary, and 10 times faster if we useLRSLibrary.However, the improvement in speed is way lower comparing to Java implementation in the paper. There are quite a few room for improving the efficiency. I am still working on this.
Detailed docstring and tests will be added in recent.
I really appreciate for any comments and suggestions.
@oyamad @QBatista @cc7768 @sglyon