Add support for development branch?

[xzilla]

I'm wondering about the idea of adding support for the development branch (currently postgres 12).

One issue is that you need to make this development version available. One option would be to force a "development branch" version into the versions hash after it does the lookup to the default source repo. This feels a bit ugly though especially in cases where people may have pointed to a custom source directory where they don't expect this other version to appear.

Another issue is how to build it. We could build out machinery to pull from git, which might make the most sense, since that way people could rebuild the dev version anytime they saw a new commit they were interested in. The downside, of course, is you need to add a bunch of routines to compile everything. It might be possible to download a zip file directly from GitHub, but that would still take some working out. If we want to leverage the existing build code, the easiest way would be to force dev to come from https://ftp.postgresql.org/pub/snapshot/dev/, although I think that is only built nightly, not per commit.

Or maybe this is all a bad idea? Thoughts?

[fluca1978]

It sounds interesting, but I see a lot of things to think about.
I think build command must be extended with a dev possibile version (and an optional commit hash, that will default to HEAD), that will pull via git the latest source code. This requires dev to be listed as a valid version number (so all the checks against a version number must be changed).
After pulling, we can jump to the configure-make-install part of the build process and have it built, but what about configuration? There will be one single dev configuration at time.
Then there's the problem of initdb, should we rebuild the PGDATA or keep the possible existing one?
What about a more general rebuild command to allow for rebuilding of an existing version (which could be dev or a "good" one) and will rebuild executables without destroying PGDATA?

However, I would not touch available because development versions should not appear there, but could be passed to build. That's the part to change in my opinion.

[fluca1978]

Thinking after a coffee, it could be worth adding a kind of build-from-git command that can accept the repo and the commit to build against, defaulting to the PostgreSQL repository and HEAD.
This is a more general approach, but all the consideration already expressed (configuration, PGDATA management and others) remain the same.

[theory]

Yeah, I like the idea of a pgenv build-dev $refspec command or something.

[fluca1978]

I've implemented a first possible approach to the problem. See pull request #26 At the moment I'm not able to complete the build process on my machine for a docbook problem, but seems correct to me.
@xzilla can you test and see if that matches your needs?

Probably there is the need for some tweaking around the git checkout and fetch process.

[e7e6]

I am rather late to the party and did not notice this issue at first.
One way to avoid an extra git dependency would be to use curl this way:
curl -L https://api.github.com/repos/postgres/postgres > postgres_master.tar.gz.
I am also interested in this feature but did not have a look yet on how @fluca1978 implemented it and what is needed to make it work.

[fluca1978]

@e7e6 while avoiding git dependency could sound good, I think that for this particular feature it is better to keep git working, since it could help "upgrading" the code base instead of having to download the whole source tree every time.

[andyatkinson]

@fluca1978 @xzilla I'm interested in this! Wondering if it was completed at some point in time in the past or was still a WIP? I'd be available to help test compilation on Mac and sliding those commands into shell scripts, if it helps. My ideal DX would be something like pgenv build snapshot-dev (this doesn't exist) or something that builds from: https://ftp.postgresql.org/pub/snapshot/dev/

Or maybe running compilation steps from my local git checkout, like pgenv build local (this doesn't exist). Wondering if this would be like RUBY_BUILD_BUILD_PATH from ruby-build, which is what rbenv uses for Ruby sources. https://github.com/rbenv/ruby-build

[fluca1978]

@fluca1978 @xzilla I'm interested in this! Wondering if it was completed at some point in time in the past or was still a WIP? I'd be available to help test compilation on Mac and sliding those commands into shell scripts, if it helps. My ideal DX would be something like pgenv build snapshot-dev (this doesn't exist) or something that builds from: https://ftp.postgresql.org/pub/snapshot/dev/

Or maybe running compilation steps from my local git checkout, like pgenv build local (this doesn't exist). Wondering if this would be like RUBY_BUILD_BUILD_PATH from ruby-build, which is what rbenv uses for Ruby sources. https://github.com/rbenv/ruby-build

Well, so far we have the chance to compile from a local repostiory by means of setting the environment variable PGENV_LOCAL_POSTGRESQL_REPO, that will literally go into the local repostiroy and build from there.
It would be interesting to have a build-from like command that can accept a repository URL and an optional tag/branch.

[theory]

I tried this last week, by coincidence. It already uses tags, so if you tell it to build 16.2, it knows to build REL_16_2. It doesn't know how to build a branch or master, though, because it complains if the version you give it is clearly not a version.

To get this to work, I think we'd need loosen the version checking when PGENV_LOCAL_POSTGRESQL_REPO is set to check to see if the named git reference exists.

[pjungwir]

I'm interested in this feature. It would help me develop extensions that build on not-yet-released functionality (like SupportRequestInlineInFrom).

As you say, we already have PGENV_LOCAL_POSTGRESQL_REPO. What do you think of adding a PGENV_GIT_REF envvar that will override the existing logic to set tag? I'd be happy to send you a PR for that. (In that case I will rename tag to ref too.)

It means you can name the result whatever you want; it isn't coupled to the tag name. That seems good & bad. You could build against REL8_3_0 and call it 18.3. But I think I like the flexibility there.

To get this to work, I think we'd need loosen the version checking when PGENV_LOCAL_POSTGRESQL_REPO is set to check to see if the named git reference exists.

Yes. I see that pgenv_input_version_or_exit already has a fourth parameter to fallback to non-version names, but that seems to be for existing installations (when switching or removing), not for building new ones. For this feature I think instead of using that we'd just check if both PGENV_LOCAL_POSTGRESQL_REPO and PGENV_GIT_REF are set, and in that case make sure that the ref exists.

I wonder if anything else would break if we supported named versions like aio or inlineinfrom---or at least 19aoi and 19inlineinfrom. I guess the latter should be safe, since we already support 18beta1 and 18rc1.

In the meantime, a workaround is to tag your local repo with a made-up future version like REL_99_0.