Draft
Conversation
It's a non-trivial task. Solution: make a proposal to address its problems
Solution: use the one I intended I meant that migrations always have to converge into a linear form
While it works, it is confusing – migration is meant to be about going from A to B. Solution: rename migration to revision
Solution: fix or rephrase
Solution: use yaml everywhere Not that this is highly critical at this point, as formats may change
Solution: provide some clarity
diogob
reviewed
Feb 11, 2025
| data/metadata.yaml | ||
| ``` | ||
|
|
||
| ***data/metadata.yaml**: |
There was a problem hiding this comment.
Do we want to use some implicit value, such as disregarding tables that are not listed? In other words, would it contain an exhaustive list of tables with data?
Contributor
Author
There was a problem hiding this comment.
Good question. Implicit is not great (defaults can change). What if we always listed all tables' preferences? Would that be okay? Gives perfect understanding as to where things should stand.
diogob
reviewed
Feb 11, 2025
| | Tar | Easy to open using command-line tools | Not directly understood by Postgres unless copied through a program; uncompressed | | ||
| | Tar.(gz/bz2/...) | Same as tar + Smaller size | Same as tar | | ||
| | Zip | Same as tar + Smaller size | Same as tar | | ||
| | SQL | Can be loaded by sending into Postgres | Can modify arbitrary objects, not directly usable in command-line for file extraction | |
There was a problem hiding this comment.
The SQL format also seems to be more interesting when using the VCS diff between commits.
Contributor
Author
There was a problem hiding this comment.
there won't be diffs here, as every file is essentially final. I also forgot to add SQLite option here, will add.
There was a problem hiding this comment.
true, I was mistakenly thinking about diffing between versions (but that would be between files in different revision directories). The point is still valid though, you could generalize Can be loaded by sending into Postgres by saying Easier to inspect.
Solution: add sqlite
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 12, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 12, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 12, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 12, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 12, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 12, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 12, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 12, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 12, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 13, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 13, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 15, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 15, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 16, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 16, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 16, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 17, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 17, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 24, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 24, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 24, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 24, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 24, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 25, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 25, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to yrashk/omnigres
that referenced
this pull request
Feb 25, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
yrashk
added a commit
to omnigres/omnigres
that referenced
this pull request
Feb 27, 2025
Tracking schema evolution, especially in presence of data-native functions (fka _stored procedures_) is a non-trivial task: * How do we prevent data and auxiliary information (such as presence of indices) loss? * How do we ensure that schema is evolved as intended? * How do we make sure the correct set of extensions at correct versions is used for every specific version? * How do we sure data-native functions correspond the correct revision of the schema? * and so on. Solution: implement a robust tracking solution following [RFC0001](omnigres/rfc#1)
Contributor
Author
|
Let's see how this RFC aged against our current implementations and discoveries. |
Contributor
Author
|
I suspect capturing migrations is not the best idea. It is complicated and makes it difficult to track relationships between previous and new generations. What if simply capture the schema in the revision? If one wants to just deploy it, it's easy – no iteration over migrations, simply restore. So every capture is a snapshot of all things. We can diff them at a different time. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
It's a non-trivial task.
Solution: make a proposal to address its problems