Options like sparse mode

[jjangga0214]

Hi.

GitHub is internally evaluating partial-clone (Actually GitLab already provides the feature!).

This can be useful for a large monorepo with sparse-checkout(see git sparse-checkout --help) feature.

Thus, hereby I request support to those features.

Thanks.

workaround

Here is a workaround suggestion for folks who want to customize the whole process, though I haven't tested.

$ REPO="https://${GITHUB_ACTOR}:${GITHUB_TOKEN}@github.com/${GITHUB_REPOSITORY}.git"

$ git clone --your-options $REPO

$ git checkout --your-options <your-subcommand> 

[annulen]

FWIW, Travis CI also supports sparse checkout, described at https://docs.travis-ci.com/user/customizing-the-build#git-sparse-checkout

It would be even better to have full-fledged partial-clone though.

[ericsciple]

actions/checkout@v2 is optimized to fetch only a single commit by default, so you should get some of the same benefits (no unnecessary history)

However doesn't solve the files-outside-of-the-user’s-work-area-in-the-tree problem.

[annulen]

It might be a good idea to add no-checkout option for action, so that people could customize checkout command directly without reimplementing initial setup

[annulen]

For now I'm using this

    - name: Sparse checkout
      shell: bash
      run: |
        REPO="https://${GITHUB_ACTOR}:${GITHUB_TOKEN}@github.com/${GITHUB_REPOSITORY}.git"
        BRANCH="${GITHUB_REF/#refs\/heads\//}"

        # Following code is based on logs of actions/checkout@v, with sparseCheckout stuff inserted in the middle
        echo "Syncing repository: $GITHUB_REPOSITORY"
        echo "Working directory is '$(pwd)' GITHUB_WORKSPACE=$GITHUB_WORKSPACE BRANCH=$BRANCH"
        git version
        git init $GITHUB_WORKSPACE
        git remote add origin https://github.com/$GITHUB_REPOSITORY
        git config --local gc.auto 0
        # Now interesting part
        git config core.sparseCheckout true
        # Add here contents of sparse-checkout line by line
        echo "..." >> .git/info/sparse-checkout
        # echo ... 
        git -c protocol.version=2 fetch --no-tags --prune --progress --depth=10 origin +${GITHUB_SHA}:refs/remotes/origin/${BRANCH}
        git checkout --progress --force -B $BRANCH refs/remotes/origin/$BRANCH

Not ideal but still saves quite a bit of time in my case

[dscho]

Not sure whether you are aware of the partial clone feature? It is still somewhat experimental, and I would be loathe to risk overloading our servers (especially during COVID-19, when we're running at or over capacity, human-wise), but maybe it would be worth playing with it for a few moments (in coordination with @github/git-core and @github/git-systems, maybe)?

[annulen]

In git 2.26 partial clone does not play well with --depth, causing massive regression

[dscho]

In git 2.26 partial clone does not play well with --depth, causing massive regression

Right, I saw some discussion on the Git mailing list, but I wasn't able to monitor that (COVID-19 🌧️).

The thing is: typically the bulk of the payload consists of blobs. The trees and commits are usually pretty light-weight. In other words, a non-shallow partial clone (that is then populated via a sparse checkout) can be a lot faster than a shallow clone. At least that's what our friends over at Google reported internally.

[annulen]

Right, I saw some discussion on the Git mailing list

And I was the one who brought the topic

non-shallow partial clone (that is then populated via a sparse checkout) can be a lot faster than a shallow clone

I guess it may be true for some class of repositories, but definitely doesn't hold in my case.

[annulen]

In the context of actions/checkout I think it would be better to provide settings controlling checkout patterns and --filter, just like it allows controlling depth now

[dscho]

Right, I saw some discussion on the Git mailing list

And I was the one who brought the topic

Whoops, I did not realize that you were the one, sorry ;-)

[fawazahmed0]

This is what I did to get partial clone:

      - name: Partial Clone
        run: |
          REPO="https://${GITHUB_ACTOR}:${{ secrets.GITHUB_TOKEN }}@github.com/${GITHUB_REPOSITORY}.git"
          git clone --filter=blob:none --no-checkout --depth 1  --sparse $REPO .
          git sparse-checkout init --cone
          git sparse-checkout add "folder1" "folder2/folder3"
          git checkout

Incase you want more power over what gets cloned, you might want to avoid the cone option

Refer this and this for more details

[dscho]

          git config --global user.email github-actions@github.com
          git config --global user.name github-actions

You don't really need to set the user.email or user.name, right?

[fawazahmed0]

          git config --global user.email github-actions@github.com
          git config --global user.name github-actions

You don't really need to set the user.email or user.name, right?

Yeah it's optional, only required if you want to commit and push the changes

[dscho]

Yeah it's optional, only required if you want to commit and push the changes

But shouldn't that be configured if/when a commit is to be created, rather than already during the checkout?

[fawazahmed0]

Yeah it's optional, only required if you want to commit and push the changes

But shouldn't that be configured if/when a commit is to be created, rather than already during the checkout?

Yep, you are right, I have updated the text, it was only relevant in my case, as I wanted to push the changes as well