hanzoai/pack
2
0
Fork 0
mirror of https://github.com/hanzoai/pack.git synced 2026-09-30 23:43:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: respect dockerignore on initial source load (#373)

Browse Source 
* docs: add claude ref to agents

* refactor: simplify DockerignoreContext passing and make it required

* refactor: another take at fixing the initial docker ignore load stuff

* style: whitespace formatting tweaks

I'm not sure how it got into this state :/

* fix: apply dockerignore rules when copying local app context, not later in the build process

* fix: merge dockerignore excludes with top-level railpack.json excludes/includes

* test: top-level railpack.json exclude/include

* docs: update excluding files docs

* test: add negation to an existing dockerignore to expand test coverage

* fix: hack to determine if I'm understanding the llb exclude functionality right

* refactor: remove Includes and instead use negation operators and Excludes

* test: respect dockerignore on initial source load

- Apply exclude and include filters during the initial buildkit context load to improve local build performance
- Support top-level exclude and include configurations in railpack.json

Generated-by: aiautocommit

* feat: respect dockerignore on initial source load

- Load local source layers using the plan package directly to ensure .dockerignore patterns are respected at the buildkit level.
- Prevent unnecessary loading of ignored files like node_modules and virtual environments during local builds.

Generated-by: aiautocommit

* style: formatting updates

* docs(config): update documentation for dockerignore and exclude pattern handling

Generated-by: aiautocommit

* feat(plan): respect dockerignore on initial source load

Generated-by: aiautocommit

* refactor(core): access include rules directly from layer struct

Generated-by: aiautocommit

* feat(buildkit): respect exclude patterns on initial source load

- Apply exclude patterns at the buildkit level during local context loading to prevent transferring unnecessary large directories like node_modules or virtual environments.

Generated-by: aiautocommit

* feat: respect dockerignore on initial source load

Generated-by: aiautocommit

* feat(core): respect dockerignore on initial source load

- Expose the parsed dockerignore context via the build generate context
- Ensure build results preserve the correct dockerignore metadata and excludes on initial load

Generated-by: aiautocommit

* style(docs): fix trailing whitespace in CLAUDE.md

Generated-by: aiautocommit

* feat(core): respect dockerignore on initial source load

- Apply exclude and include filters during buildkit context setup to avoid loading large ignored directories locally.
- Support top-level exclude and include configuration in railpack.json.

Generated-by: aiautocommit

* test(plan): assert all expected empty dockerignore examples are found

Generated-by: aiautocommit

* docs(config): fix typo in excluding-files guide

Generated-by: aiautocommit

* feat(dockerignore): respect dockerignore on initial source load

- Apply exclude and include filters during Buildkit context establishment to drastically improve local build performance.
- Support top-level `exclude` and `include` rules directly in `railpack.json`.

Generated-by: aiautocommit

* docs(config): clarify directory matching behavior in excluding-files guide

Generated-by: aiautocommit

* refactor(core): remove dead dockerignore context fields

- Remove unused dockerignore context properties from build plans and options
- Clean up obsolete metadata checks in core tests

Generated-by: aiautocommit

* feat: add top-level exclude option to railpack config

Generated-by: aiautocommit
This commit is contained in:
Michael Bianco
2026-07-31 07:41:35 -06:00
committed by GitHub
parent 012d2c2f10
commit 392d3fef96
39 changed files with 381 additions and 232 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+1
View File
@@ -0,0 +1 @@
@CLAUDE.md
CLAUDE.md
+1 -1
View File
@@ -46,7 +46,7 @@ Follow these instructions carefully when writing code:
- Don't run tests manually using `go test` unless instructed to do so
- If tests are failing that are unrelated to your changes, let me know and stop working.
- Use the `cli` mise task to test your changes on a specific example project, i.e. `mise run cli -- --verbose build --show-plan examples/node-vite-react-router-spa/`
- Run before finishing your work `mise run test-update-snapshots` and review the (possibly) updated snapshot files to make sure there are no unexpected changes.
- Run before finishing your work `mise run test-update-snapshots` and review the (possibly) updated snapshot files to make sure there are no unexpected changes.
- Do not run any write operations with `git`
- Do not use `bin/railpack` instead use `mise run cli` (which is the development build of `railpack`)
- Therefore do not run `mise build`, we don't need a `railpack` binary for local testing
buildkit/convert.go
+10 -4
View File
@@ -28,7 +28,6 @@ type ConvertPlanOptions struct {
// Token used to make authenticated API requests to GitHub to increase rate limits
GitHubToken string
// Do not use cache when building
NoCache bool
}
@@ -40,12 +39,19 @@ const (
func ConvertPlanToLLB(plan *p.BuildPlan, opts ConvertPlanOptions) (*llb.State, *Image, error) {
platform := opts.BuildPlatform
localState := llb.Local("context",
// by default, the whole directory is transferred into context, we don't need to explicitly include it
localOpts := []llb.LocalOption{
llb.SharedKeyHint("local"),
llb.SessionID(opts.SessionID),
llb.WithCustomName("loading ."),
llb.FollowPaths([]string{"."}),
)
}
// note that exclude patterns can contain inverse (inclusions) patterns. The llb.IncludePatterns should *not* be used for this
if len(plan.Exclude) > 0 {
localOpts = append(localOpts, llb.ExcludePatterns(plan.Exclude))
}
localState := llb.Local("context", localOpts...)
cacheStore := build_llb.NewBuildKitCacheStore(opts.CacheKey)
graph, err := build_llb.NewBuildGraph(plan, &localState, cacheStore, opts.SecretsHash, &platform, opts.GitHubToken, opts.NoCache)
core/__snapshots__/TestGenerateBuildPlanForExamples_dockerignore_1.snap.json
+21 -18
View File
@@ -16,6 +16,26 @@
"RAILPACK_VERSION": "dev"
}
},
"exclude": [
".vscode",
".copier",
".env-specific",
".env*",
"__pycache__",
"test",
"tmp/*",
"*.log",
"Justfile",
"TODO*",
"README.md",
"docker-compose*.yml",
"negation_test/*",
"!negation_test/should_exist.txt",
"!negation_test/should_not_exist.txt",
"!negation_test/folder_does_not_exist",
"!negation_test/existing_folder",
"**/node_modules"
],
"steps": [
{
"inputs": [
@@ -36,25 +56,8 @@
"step": "packages:mise"
},
{
"exclude": [
".vscode",
".copier",
".env-specific",
".env*",
"__pycache__",
"test",
"tmp/*",
"*.log",
"Justfile",
"TODO*",
"README.md",
"docker-compose*.yml",
"negation_test/*"
],
"include": [
".",
"negation_test/should_exist.txt",
"negation_test/existing_folder"
"."
],
"local": true
}
core/__snapshots__/TestGenerateBuildPlanForExamples_node-vite-react-router-spa_1.snap.json
+7 -6
View File
@@ -60,6 +60,13 @@
"RAILPACK_VERSION": "dev"
}
},
"exclude": [
".react-router",
"build",
"node_modules",
"README.md",
"!app"
],
"steps": [
{
"assets": {
@@ -155,12 +162,6 @@
"step": "install"
},
{
"exclude": [
".react-router",
"build",
"node_modules",
"README.md"
],
"include": [
"."
],
core/__snapshots__/TestGenerateBuildPlanForExamples_node-vite-react-router-ssr_1.snap.json
+6 -6
View File
@@ -69,6 +69,12 @@
"RAILPACK_VERSION": "dev"
}
},
"exclude": [
".react-router",
"build",
"node_modules",
"README.md"
],
"steps": [
{
"assets": {
@@ -164,12 +170,6 @@
"step": "install"
},
{
"exclude": [
".react-router",
"build",
"node_modules",
"README.md"
],
"include": [
"."
],
core/config/config.go
+1
View File
@@ -34,6 +34,7 @@ type Config struct {
Packages map[string]string `json:"packages,omitempty" jsonschema:"description=Map of package name to package version"`
Caches map[string]*plan.Cache `json:"caches,omitempty" jsonschema:"description=Map of cache name to cache definitions. The cache key can be referenced in an exec command"`
Secrets []string `json:"secrets,omitempty" jsonschema:"description=Secrets that should be made available to commands that have useSecrets set to true"`
Exclude []string `json:"exclude,omitempty" jsonschema:"description=File patterns to exclude from the build context (alternative to .dockerignore). Supports negation patterns starting with ! to include files that would otherwise be excluded"`
}
func EmptyConfig() *Config {
core/config/config_test.go
+45
View File
@@ -332,3 +332,48 @@ func TestGetJsonSchema(t *testing.T) {
require.NoError(t, err)
require.NotEmpty(t, schemaJson)
}
func TestConfigExcludeInclude(t *testing.T) {
configJSON := `{
"exclude": [
"node_modules",
"*.log",
".env",
"!important.log"
]
}`
var config Config
require.NoError(t, json.Unmarshal([]byte(configJSON), &config))
require.Equal(t, []string{"node_modules", "*.log", ".env", "!important.log"}, config.Exclude)
}
func TestMergeConfigExcludeInclude(t *testing.T) {
config1JSON := `{
"exclude": ["node_modules", ".env", "!important.log"]
}`
config2JSON := `{
"exclude": ["*.tmp", "!keep.tmp"]
}`
expectedJSON := `{
"exclude": ["*.tmp", "!keep.tmp"],
"steps": {},
"packages": {},
"caches": {},
"deploy": {}
}`
var config1, config2, expected Config
require.NoError(t, json.Unmarshal([]byte(config1JSON), &config1))
require.NoError(t, json.Unmarshal([]byte(config2JSON), &config2))
require.NoError(t, json.Unmarshal([]byte(expectedJSON), &expected))
result := Merge(&config1, &config2)
if diff := cmp.Diff(expected, *result); diff != "" {
t.Errorf("configs mismatch (-want +got):\n%s", diff)
}
}
core/core_test.go
+1
View File
@@ -133,4 +133,5 @@ func TestGenerateBuildPlan_DockerignoreMetadata(t *testing.T) {
require.True(t, buildResult.Success)
require.NotNil(t, buildResult.Metadata)
require.Equal(t, "true", buildResult.Metadata["dockerIgnore"])
require.NotEmpty(t, buildResult.Plan.Exclude)
}
core/generate/context.go
+10 -18
View File
@@ -57,6 +57,7 @@ type CommandWrapper struct {
Command plan.Command
}
// is a user-provided command entry a "spread" command?
func (c CommandWrapper) IsSpread() bool {
if execCmd, ok := c.Command.(plan.ExecCommand); ok {
return execCmd.Cmd == plan.ShellCommandString("...") || execCmd.Cmd == "..."
@@ -77,9 +78,7 @@ func NewGenerateContext(app *a.App, env *a.Environment, config *config.Config, l
if dockerignoreCtx.HasFile {
logger.LogInfo("Found .dockerignore file, applying filters")
log.Debugf("Dockerignore exclude patterns: %v", dockerignoreCtx.Excludes)
log.Debugf("Dockerignore include patterns: %v", dockerignoreCtx.Includes)
log.Debugf("Dockerignore patterns: %v", dockerignoreCtx.Excludes)
}
ctx := &GenerateContext{
@@ -153,9 +152,16 @@ func (c *GenerateContext) Generate() (*plan.BuildPlan, map[string]*resolver.Reso
return nil, nil, err
}
// Create the actual build plan
buildPlan := plan.NewBuildPlan()
// Merge exclude patterns from .dockerignore and railpack.json
excludePatterns := []string{}
excludePatterns = append(excludePatterns, c.dockerignoreCtx.Excludes...)
excludePatterns = append(excludePatterns, c.Config.Exclude...)
if len(excludePatterns) > 0 {
buildPlan.Exclude = excludePatterns
}
buildStepOptions := &BuildStepOptions{
ResolvedPackages: resolvedPackages,
Caches: c.Caches,
@@ -302,20 +308,6 @@ func (c *GenerateContext) applyDeployAptPackages() {
c.Deploy.AptPackages = plan.SpreadStrings(configuredPackages, c.Deploy.AptPackages)
}
// creates a local layer with dockerignore patterns applied
func (c *GenerateContext) NewLocalLayer() plan.Layer {
layer := plan.NewLocalLayer()
if len(c.dockerignoreCtx.Includes) > 0 {
layer.Include = append(layer.Include, c.dockerignoreCtx.Includes...)
}
if len(c.dockerignoreCtx.Excludes) > 0 {
layer.Exclude = append(layer.Exclude, c.dockerignoreCtx.Excludes...)
}
return layer
}
// in order to get around a circular dependency issue, we need to define discrete getters to interface with
// the mise package version logic.
core/generate/context_test.go
+28 -21
View File
@@ -4,6 +4,7 @@ import (
"encoding/json"
"os"
"path/filepath"
"slices"
"testing"
"github.com/gkampitakis/go-snaps/snaps"
@@ -204,6 +205,18 @@ func TestGenerateContextAppliesConfiguredDeployBase(t *testing.T) {
}
func TestGenerateContextDockerignore(t *testing.T) {
t.Run("dockerignore patterns precede config patterns", func(t *testing.T) {
ctx := CreateTestContext(t, "../../examples/dockerignore")
configExcludes := []string{"!the.log", "config-only"}
ctx.Config.Exclude = configExcludes
buildPlan, _, err := ctx.Generate()
require.NoError(t, err)
expected := append(slices.Clone(ctx.dockerignoreCtx.Excludes), configExcludes...)
require.Equal(t, expected, buildPlan.Exclude)
})
t.Run("context with dockerignore", func(t *testing.T) {
ctx := CreateTestContext(t, "../../examples/dockerignore")
@@ -213,19 +226,23 @@ func TestGenerateContextDockerignore(t *testing.T) {
// Verify metadata indicates dockerignore presence
require.Equal(t, "true", ctx.Metadata.Get("dockerIgnore"))
// Test NewLocalLayer with dockerignore patterns
layer := ctx.NewLocalLayer()
require.True(t, layer.Local)
require.NotNil(t, layer.Filter)
// Verify dockerignore patterns are in the context
require.NotEmpty(t, ctx.dockerignoreCtx.Excludes)
require.Contains(t, ctx.dockerignoreCtx.Excludes, ".vscode")
require.Contains(t, ctx.dockerignoreCtx.Excludes, "*.log")
require.Contains(t, ctx.dockerignoreCtx.Excludes, "__pycache__")
// Should have exclude patterns from .dockerignore
require.NotEmpty(t, layer.Exclude)
require.Contains(t, layer.Exclude, ".vscode")
require.Contains(t, layer.Exclude, "*.log")
require.Contains(t, layer.Exclude, "__pycache__") // Trailing slash is stripped by parser
// Negation patterns are also in Excludes (with ! prefix)
require.Contains(t, ctx.dockerignoreCtx.Excludes, "!negation_test/should_exist.txt")
require.Contains(t, ctx.dockerignoreCtx.Excludes, "!negation_test/existing_folder")
// Should have default include pattern
require.Equal(t, []string{".", "negation_test/should_exist.txt", "negation_test/existing_folder"}, layer.Include)
// Verify dockerignore patterns are correctly moved to the plan level
buildPlan, _, err := ctx.Generate()
require.NoError(t, err)
require.Contains(t, buildPlan.Exclude, ".vscode")
require.Contains(t, buildPlan.Exclude, "*.log")
require.Contains(t, buildPlan.Exclude, "!negation_test/should_exist.txt")
require.Contains(t, buildPlan.Exclude, "!negation_test/existing_folder")
})
t.Run("context without dockerignore", func(t *testing.T) {
@@ -236,15 +253,6 @@ func TestGenerateContextDockerignore(t *testing.T) {
// Verify metadata does not indicate dockerignore presence
require.Empty(t, ctx.Metadata.Get("dockerIgnore"))
// Test NewLocalLayer without dockerignore patterns
layer := ctx.NewLocalLayer()
require.True(t, layer.Local)
// Should use default behavior when no dockerignore patterns exist
require.NotNil(t, layer.Filter)
require.Equal(t, []string{"."}, layer.Include)
require.Empty(t, layer.Exclude)
})
t.Run("context creation with no dockerignore", func(t *testing.T) {
@@ -257,7 +265,6 @@ func TestGenerateContextDockerignore(t *testing.T) {
// Verify parsing works with no file present
require.Nil(t, ctx.dockerignoreCtx.Excludes)
require.Nil(t, ctx.dockerignoreCtx.Includes)
})
t.Run("context creation fails with invalid dockerignore", func(t *testing.T) {
core/plan/dockerignore.go
+12 -46
View File
@@ -4,79 +4,45 @@ import (
"strings"
"github.com/railwayapp/railpack/core/app"
"github.com/railwayapp/railpack/internal/utils"
// this is the native dockerignore parser used by buildkit
// https://github.com/moby/buildkit/blob/master/frontend/dockerfile/dockerignore/dockerignore_deprecated.go
"github.com/moby/patternmatcher/ignorefile"
)
// checks if a .dockerignore file exists in the app directory and parses it
func CheckAndParseDockerignore(app *app.App) ([]string, []string, error) {
if !app.HasFile(".dockerignore") {
return nil, nil, nil
}
// parses a .dockerignore file from the app directory. assumes the file exists.
func checkAndParseDockerignore(app *app.App) ([]string, error) {
content, err := app.ReadFile(".dockerignore")
if err != nil {
return nil, nil, err
return nil, err
}
reader := strings.NewReader(content)
patterns, err := ignorefile.ReadAll(reader)
if err != nil {
return nil, nil, err
return nil, err
}
excludePatterns, includePatterns := separatePatterns(patterns)
// Filter include patterns that don't match any files
// This prevents build errors when a negated pattern in .dockerignore refers to a missing file
var validIncludes []string
for _, pattern := range includePatterns {
if app.HasMatch(pattern) {
validIncludes = append(validIncludes, pattern)
}
}
return excludePatterns, validIncludes, nil
}
// separatePatterns separates patterns into exclude and include lists
// Include patterns are those starting with '!' (negation)
func separatePatterns(patterns []string) (excludes []string, includes []string) {
for _, pattern := range patterns {
if len(pattern) > 0 && pattern[0] == '!' {
// Remove the '!' prefix for include patterns
includes = append(includes, pattern[1:])
} else {
excludes = append(excludes, pattern)
}
}
return excludes, includes
return patterns, nil
}
type DockerignoreContext struct {
Excludes []string
Includes []string
HasFile bool
}
func NewDockerignoreContext(app *app.App) (*DockerignoreContext, error) {
hasFile := app.HasFile(".dockerignore")
excludes, includes, err := CheckAndParseDockerignore(app)
if !app.HasFile(".dockerignore") {
return &DockerignoreContext{}, nil
}
excludes, err := checkAndParseDockerignore(app)
if err != nil {
return nil, err
}
if excludes != nil {
excludes = utils.RemoveDuplicates(excludes)
}
if includes != nil {
includes = utils.RemoveDuplicates(includes)
}
return &DockerignoreContext{
Excludes: excludes,
Includes: includes,
HasFile: hasFile,
HasFile: true,
}, nil
}
core/plan/dockerignore_test.go
+62 -84
View File
@@ -3,6 +3,7 @@ package plan
import (
"os"
"path/filepath"
"slices"
"testing"
"github.com/railwayapp/railpack/core/app"
@@ -10,32 +11,17 @@ import (
)
func TestCheckAndParseDockerignore(t *testing.T) {
t.Run("nonexistent dockerignore", func(t *testing.T) {
tempDir, err := os.MkdirTemp("", "dockerignore-test")
require.NoError(t, err)
defer func() { _ = os.RemoveAll(tempDir) }()
testApp, err := app.NewApp(tempDir)
require.NoError(t, err)
excludes, includes, err := CheckAndParseDockerignore(testApp)
require.NoError(t, err)
require.Nil(t, excludes)
require.Nil(t, includes)
})
t.Run("valid dockerignore file", func(t *testing.T) {
examplePath := filepath.Join("..", "..", "examples", "dockerignore")
testApp, err := app.NewApp(examplePath)
require.NoError(t, err)
excludes, includes, err := CheckAndParseDockerignore(testApp)
patterns, err := checkAndParseDockerignore(testApp)
require.NoError(t, err)
require.NotNil(t, excludes)
require.NotNil(t, includes)
require.Contains(t, includes, "negation_test/should_exist.txt")
require.Contains(t, includes, "negation_test/existing_folder")
require.NotNil(t, patterns)
require.Contains(t, patterns, "!negation_test/should_exist.txt")
require.Contains(t, patterns, "!negation_test/existing_folder")
// Verify some expected patterns from examples/dockerignore/.dockerignore
// Note: patterns are parsed by the moby/patternmatcher library
@@ -55,7 +41,7 @@ func TestCheckAndParseDockerignore(t *testing.T) {
}
for _, expected := range expectedPatterns {
require.Contains(t, excludes, expected, "Expected pattern %s not found in excludes", expected)
require.Contains(t, patterns, expected, "Expected pattern %s not found in patterns", expected)
}
})
@@ -78,54 +64,56 @@ func TestCheckAndParseDockerignore(t *testing.T) {
require.NoError(t, err)
// This should fail with a permission error
excludes, includes, err := CheckAndParseDockerignore(testApp)
patterns, err := checkAndParseDockerignore(testApp)
require.Error(t, err)
require.Contains(t, err.Error(), "error reading .dockerignore")
require.Nil(t, excludes)
require.Nil(t, includes)
require.Nil(t, patterns)
})
}
func TestSeparatePatterns(t *testing.T) {
t.Run("only exclude patterns", func(t *testing.T) {
patterns := []string{"*.log", "node_modules", "/tmp"}
excludes, includes := separatePatterns(patterns)
func TestDockerignoreExamples(t *testing.T) {
wd, err := os.Getwd()
require.NoError(t, err)
require.Equal(t, patterns, excludes)
require.Empty(t, includes)
})
examplesDir := filepath.Join(filepath.Dir(filepath.Dir(wd)), "examples")
entries, err := os.ReadDir(examplesDir)
require.NoError(t, err)
t.Run("only include patterns", func(t *testing.T) {
patterns := []string{"!important.log", "!keep/this"}
excludes, includes := separatePatterns(patterns)
expectedAsEmpty := []string{"mise-config"}
foundAsEmpty := []string{}
testedExamples := 0
for _, entry := range entries {
if !entry.IsDir() {
continue
}
require.Empty(t, excludes)
require.Equal(t, []string{"important.log", "keep/this"}, includes)
})
examplePath := filepath.Join(examplesDir, entry.Name())
dockerignorePath := filepath.Join(examplePath, ".dockerignore")
_, err := os.Stat(dockerignorePath)
if os.IsNotExist(err) {
continue
}
require.NoError(t, err)
t.Run("mixed patterns", func(t *testing.T) {
patterns := []string{"*.log", "!important.log", "node_modules", "!node_modules/keep"}
excludes, includes := separatePatterns(patterns)
testedExamples++
t.Run(entry.Name(), func(t *testing.T) {
testApp, err := app.NewApp(examplePath)
require.NoError(t, err)
require.Equal(t, []string{"*.log", "node_modules"}, excludes)
require.Equal(t, []string{"important.log", "node_modules/keep"}, includes)
})
ctx, err := NewDockerignoreContext(testApp)
require.NoError(t, err)
require.True(t, ctx.HasFile)
if slices.Contains(expectedAsEmpty, entry.Name()) {
foundAsEmpty = append(foundAsEmpty, entry.Name())
require.Empty(t, ctx.Excludes)
return
}
require.NotEmpty(t, ctx.Excludes)
})
}
t.Run("empty patterns", func(t *testing.T) {
patterns := []string{}
excludes, includes := separatePatterns(patterns)
require.Empty(t, excludes)
require.Empty(t, includes)
})
t.Run("empty string patterns", func(t *testing.T) {
patterns := []string{"", "*.log", "", "!keep.log"}
excludes, includes := separatePatterns(patterns)
require.Equal(t, []string{"", "*.log", ""}, excludes)
require.Equal(t, []string{"keep.log"}, includes)
})
require.Positive(t, testedExamples, "expected at least one example with a .dockerignore")
require.ElementsMatch(t, expectedAsEmpty, foundAsEmpty)
}
func TestDockerignoreContext(t *testing.T) {
@@ -142,7 +130,6 @@ func TestDockerignoreContext(t *testing.T) {
require.NotNil(t, ctx)
require.False(t, ctx.HasFile)
require.Nil(t, ctx.Excludes)
require.Nil(t, ctx.Includes)
})
t.Run("context with dockerignore file", func(t *testing.T) {
@@ -154,9 +141,8 @@ func TestDockerignoreContext(t *testing.T) {
require.NoError(t, err)
require.True(t, ctx.HasFile)
require.NotNil(t, ctx.Excludes)
require.NotNil(t, ctx.Includes)
require.Contains(t, ctx.Includes, "negation_test/should_exist.txt")
require.Contains(t, ctx.Includes, "negation_test/existing_folder")
require.Contains(t, ctx.Excludes, "!negation_test/should_exist.txt")
require.Contains(t, ctx.Excludes, "!negation_test/existing_folder")
})
t.Run("parse nonexistent file", func(t *testing.T) {
@@ -171,7 +157,6 @@ func TestDockerignoreContext(t *testing.T) {
require.NoError(t, err)
require.False(t, ctx.HasFile)
require.Nil(t, ctx.Excludes)
require.Nil(t, ctx.Includes)
})
t.Run("parse error handling", func(t *testing.T) {
@@ -199,7 +184,7 @@ func TestDockerignoreContext(t *testing.T) {
}
func TestDockerignoreDuplicatePatterns(t *testing.T) {
t.Run("duplicate patterns removed", func(t *testing.T) {
t.Run("duplicate patterns not removed from raw output", func(t *testing.T) {
tempDir, err := os.MkdirTemp("", "dockerignore-test")
require.NoError(t, err)
defer func() { _ = os.RemoveAll(tempDir) }()
@@ -227,6 +212,7 @@ node_modules
// Count occurrences of each pattern
logCount := 0
nodeModulesCount := 0
keepCount := 0
for _, pattern := range ctx.Excludes {
if pattern == "*.log" {
logCount++
@@ -234,24 +220,20 @@ node_modules
if pattern == "node_modules" {
nodeModulesCount++
}
}
keepCount := 0
for _, pattern := range ctx.Includes {
if pattern == "keep.txt" {
if pattern == "!keep.txt" {
keepCount++
}
}
// Verify no duplicates exist
require.LessOrEqual(t, logCount, 1, "*.log pattern should appear at most once")
require.LessOrEqual(t, nodeModulesCount, 1, "node_modules pattern should appear at most once")
require.LessOrEqual(t, keepCount, 1, "keep.txt pattern should appear at most once")
// Duplicates are preserved in raw output (will be handled downstream)
require.Equal(t, 2, logCount, "*.log pattern should appear twice")
require.Equal(t, 1, nodeModulesCount, "node_modules pattern should appear once")
require.Equal(t, 2, keepCount, "!keep.txt pattern should appear twice")
})
}
func TestCheckAndParseDockerignoreWithNegation(t *testing.T) {
t.Run("negated patterns with existing and non-existing files and folders", func(t *testing.T) {
t.Run("negated patterns preserved in output", func(t *testing.T) {
tempDir, err := os.MkdirTemp("", "dockerignore-test")
require.NoError(t, err)
defer func() { _ = os.RemoveAll(tempDir) }()
@@ -278,18 +260,14 @@ negation_test/*
testApp, err := app.NewApp(tempDir)
require.NoError(t, err)
excludes, includes, err := CheckAndParseDockerignore(testApp)
patterns, err := checkAndParseDockerignore(testApp)
require.NoError(t, err)
// Check excludes
require.Contains(t, excludes, "negation_test/*")
// Check includes - should only contain the file/folder that actually exists
require.Contains(t, includes, "negation_test/should_exist.txt")
require.Contains(t, includes, "negation_test/existing_folder")
require.Contains(t, includes, "negation_test/existing_folder")
require.NotContains(t, includes, "negation_test/should_not_exist.txt")
require.NotContains(t, includes, "negation_test/folder_does_not_exist/")
require.Len(t, includes, 2)
// All patterns (both exclude and negated) should be preserved
require.Contains(t, patterns, "negation_test/*")
require.Contains(t, patterns, "!negation_test/should_exist.txt")
require.Contains(t, patterns, "!negation_test/should_not_exist.txt")
require.Contains(t, patterns, "!negation_test/folder_does_not_exist")
require.Contains(t, patterns, "!negation_test/existing_folder")
})
}
core/plan/plan.go
+2
View File
@@ -16,6 +16,7 @@ type BuildPlan struct {
Caches map[string]*Cache `json:"caches,omitempty"`
Secrets []string `json:"secrets,omitempty"`
Deploy Deploy `json:"deploy"`
Exclude []string `json:"exclude,omitempty"`
}
type Deploy struct {
@@ -41,6 +42,7 @@ func NewBuildPlan() *BuildPlan {
Deploy: Deploy{},
Caches: make(map[string]*Cache),
Secrets: []string{},
Exclude: []string{},
}
}
core/providers/cpp/cpp.go
+1 -1
View File
@@ -43,7 +43,7 @@ func (p *CppProvider) Plan(ctx *generate.GenerateContext) error {
build := ctx.NewCommandStep("build")
build.AddInput(plan.NewStepLayer(packages.Name()))
build.AddInput(ctx.NewLocalLayer())
build.AddInput(plan.NewLocalLayer())
build.AddCommand(plan.NewExecCommand("mkdir /build"))
buildsystem.Build(build)
core/providers/deno/deno.go
+1 -1
View File
@@ -72,7 +72,7 @@ func (p *DenoProvider) Build(ctx *generate.GenerateContext, build *generate.Comm
return
}
build.AddInput(ctx.NewLocalLayer())
build.AddInput(plan.NewLocalLayer())
build.AddCommands([]plan.Command{
plan.NewExecCommand(fmt.Sprintf("deno cache %s", p.mainFile)),
})
core/providers/dotnet/dotnet.go
+1 -1
View File
@@ -99,7 +99,7 @@ func (p *DotnetProvider) Install(ctx *generate.GenerateContext, install *generat
func (p *DotnetProvider) Build(ctx *generate.GenerateContext, build *generate.CommandStepBuilder) {
maps.Copy(build.Variables, p.GetEnvVars(ctx))
build.AddInput(ctx.NewLocalLayer())
build.AddInput(plan.NewLocalLayer())
build.AddCommands([]plan.Command{
plan.NewExecCommand("dotnet publish --no-restore -c Release -o out"),
})
core/providers/gleam/gleam.go
+1 -1
View File
@@ -28,7 +28,7 @@ func (p *GleamProvider) StartCommandHelp() string {
func (p *GleamProvider) Plan(ctx *generate.GenerateContext) error {
build := ctx.NewCommandStep("build")
build.AddInput(plan.NewStepLayer(ctx.GetMiseStepBuilder().Name()))
build.AddInput(ctx.NewLocalLayer())
build.AddInput(plan.NewLocalLayer())
build.AddCommand(plan.NewExecCommand("gleam export erlang-shipment"))
p.installErlang(ctx.GetMiseStepBuilder())
core/providers/golang/golang.go
+1 -1
View File
@@ -99,7 +99,7 @@ func (p *GoProvider) Build(ctx *generate.GenerateContext, build *generate.Comman
buildCmd = fmt.Sprintf("%s main.go", baseBuildCmd)
}
build.AddInput(ctx.NewLocalLayer())
build.AddInput(plan.NewLocalLayer())
if buildCmd == "" {
return
core/providers/java/java.go
+1 -1
View File
@@ -30,7 +30,7 @@ func (p *JavaProvider) StartCommandHelp() string {
func (p *JavaProvider) Plan(ctx *generate.GenerateContext) error {
build := ctx.NewCommandStep("build")
build.AddInput(plan.NewStepLayer(ctx.GetMiseStepBuilder().Name()))
build.AddInput(ctx.NewLocalLayer())
build.AddInput(plan.NewLocalLayer())
if p.usesGradle(ctx) {
ctx.Logger.LogInfo("Using Gradle")
core/providers/node/node.go
+1 -1
View File
@@ -265,7 +265,7 @@ func (p *NodeProvider) GetStartCommand(ctx *generate.GenerateContext) string {
}
func (p *NodeProvider) Build(ctx *generate.GenerateContext, build *generate.CommandStepBuilder) {
build.AddInput(ctx.NewLocalLayer())
build.AddInput(plan.NewLocalLayer())
if p.packageJson.HasScript("build") {
build.AddCommands([]plan.Command{
core/providers/node/package_manager.go
+1 -1
View File
@@ -100,7 +100,7 @@ func (p PackageManager) installDependencies(ctx *generate.GenerateContext, works
// If there are any pre/post install scripts, we need the entire app to be copied
// This is to handle things like patch-package
if hasPreInstall || hasPostInstall || hasPrepare || usesLocalFile {
install.AddInput(ctx.NewLocalLayer())
install.AddInput(plan.NewLocalLayer())
// Use all secrets for the install step if there are any pre/post install scripts
install.UseSecrets([]string{"*"})
core/providers/php/php.go
+1 -1
View File
@@ -91,7 +91,7 @@ func (p *PhpProvider) Plan(ctx *generate.GenerateContext) error {
// A manual build command will go here
build := ctx.NewCommandStep("build")
build.AddInput(plan.NewStepLayer(composer.Name()))
build.AddInput(ctx.NewLocalLayer())
build.AddInput(plan.NewLocalLayer())
ctx.Deploy.Base = plan.NewStepLayer(build.Name())
p.ConditionallyIncludeMise(ctx)
}
core/providers/python/python.go
+2 -2
View File
@@ -118,7 +118,7 @@ func (p *PythonProvider) Plan(ctx *generate.GenerateContext) error {
p.addMetadata(ctx)
build.AddInput(plan.NewStepLayer(install.Name()))
build.AddInput(ctx.NewLocalLayer())
build.AddInput(plan.NewLocalLayer())
ctx.Deploy.StartCmd = p.GetStartCommand(ctx)
maps.Copy(ctx.Deploy.Variables, p.GetPythonEnvVars(ctx))
@@ -434,7 +434,7 @@ func (p *PythonProvider) GetPythonEnvVars(ctx *generate.GenerateContext) map[str
func (p *PythonProvider) copyInstallFiles(ctx *generate.GenerateContext, install *generate.CommandStepBuilder) {
if p.installNeedsAllFiles(ctx) {
install.AddInput(ctx.NewLocalLayer())
install.AddInput(plan.NewLocalLayer())
return
}
core/providers/ruby/ruby.go
+1 -1
View File
@@ -186,7 +186,7 @@ func (p *RubyProvider) Build(ctx *generate.GenerateContext, build *generate.Comm
build.Secrets = []string{}
build.UseSecretsWithPrefixes([]string{"RAILS", "BUNDLE", "BOOTSNAP", "SPROCKETS", "WEBPACKER", "ASSET", "DISABLE_SPRING"})
build.AddEnvVars(p.GetRubyEnvVars(ctx))
build.AddInput(ctx.NewLocalLayer())
build.AddInput(plan.NewLocalLayer())
outputs := []string{"/app"}
// Only compile assets if a Rails app have an asset pipeline gem
// installed (e.g. sprockets, propshaft). Rails API-only apps [0]
core/providers/rust/rust.go
+1 -1
View File
@@ -193,7 +193,7 @@ func (p *RustProvider) Install(ctx *generate.GenerateContext, install *generate.
}
func (p *RustProvider) Build(ctx *generate.GenerateContext, build *generate.CommandStepBuilder) {
build.AddInput(ctx.NewLocalLayer())
build.AddInput(plan.NewLocalLayer())
build.AddCommands([]plan.Command{
plan.NewExecCommand("mkdir -p bin"),
})
core/providers/shell/shell.go
+2 -2
View File
@@ -59,7 +59,7 @@ func (p *ShellProvider) Plan(ctx *generate.GenerateContext) error {
install := ctx.NewCommandStep("install")
// Install step needs mise base to access any tools installed in the mise step (e.g. via RAILPACK_PACKAGES)
install.AddInput(plan.NewStepLayer(miseStepName))
install.AddInput(ctx.NewLocalLayer())
install.AddInput(plan.NewLocalLayer())
// If we have an install step, the build step should be based on the result of the install step
// so that artifacts from the install step are available during the build.
@@ -71,7 +71,7 @@ func (p *ShellProvider) Plan(ctx *generate.GenerateContext) error {
build := ctx.NewCommandStep("build")
build.AddInput(buildBaseLayer)
build.AddInput(ctx.NewLocalLayer())
build.AddInput(plan.NewLocalLayer())
build.AddCommands(
[]plan.Command{
plan.NewExecCommand("chmod +x " + p.scriptName),
core/providers/staticfile/staticfile.go
+1 -1
View File
@@ -58,7 +58,7 @@ func (p *StaticfileProvider) Plan(ctx *generate.GenerateContext) error {
build := ctx.NewCommandStep("build")
build.AddInput(plan.NewStepLayer(installCaddyStep.Name()))
build.AddInput(ctx.NewLocalLayer())
build.AddInput(plan.NewLocalLayer())
// the Staticfile provider is also used by Node SPA, but in that case we want index fallback to default to true
// for the Staticfile provider, want to default to false, which is why we set the default here instead of upstream
core/testing/context.go
+1
View File
@@ -1,3 +1,4 @@
// testing utilities
package testing
import (
docs/src/content/docs/config/excluding-files.md
+102 -6
View File
@@ -3,20 +3,116 @@ title: Excluding Files
description: How to exclude files from your build using .dockerignore
---
Railpack supports the standard `.dockerignore` file syntax to exclude files from your build. For a complete reference on the syntax, please refer to the [official Docker documentation](https://docs.docker.com/build/building/context/#dockerignore-files).
Railpack supports the standard `.dockerignore` file syntax to exclude files
from your build. For a complete reference on the syntax, please refer to the
[official Docker documentation](https://docs.docker.com/build/building/context/#dockerignore-files).
## How .dockerignore Works
When you use a `.dockerignore` file, Railpack parses it and passes all
patterns (including negations) to BuildKit in a single `exclude` list.
BuildKit's native pattern matching handles both exclusions and negations
(lines starting with `!`).
Unlike `.gitignore`, `cache/` and `cache` are equivalent and can exclude either
a file or a directory.
### Example
Given a `.dockerignore` file:
```
**/node_modules
.env
*.log
!important.log
```
When you run `railpack build --show-plan`, you'll see this gets converted to:
```json
{
"exclude": [
"**/node_modules",
".env",
"*.log",
"!important.log"
],
"steps": [...],
"deploy": {...}
}
```
## Using railpack.json
You can also specify `exclude` patterns directly in your `railpack.json`
configuration file instead of (or in addition to) using `.dockerignore`.
Negation patterns (starting with `!`) can be included in the `exclude` array:
```json
{
"exclude": [
"**/node_modules",
"**/.venv",
"*.log",
".env",
"!important.log"
],
"deploy": {
"startCommand": "node server.js"
}
}
```
This gives you more control and allows you to manage all build configuration in
one place. If both `.dockerignore` and `railpack.json` exclude patterns are
present, they are merged together.
## Default Behavior
If no `.dockerignore` file is present in your project root, Railpack defaults to including **all files** in the build context.
If no `.dockerignore` file is present in your project root, Railpack defaults
to including **all files** in the build context.
## Common Patterns
### Excluding Nested Directories
A common gotcha: patterns without wildcards only match at the root level.
```
# Only excludes /node_modules (root level)
node_modules
# Excludes node_modules at ANY nesting level
**/node_modules
```
For example, if you have this structure:
```
project/
├── node_modules/ ← excluded by both patterns
└── web/
└── node_modules/ ← only excluded by **/node_modules
```
Use `**/` prefix to match directories at any depth. This applies to all
directory names: `**/node_modules`, `**/.venv`, `**/vendor`, etc.
## Best Practices
### Local Development Artifacts
If you run builds locally, it is critical to exclude local environment folders like `node_modules`, `.venv`, or `vendor`.
If you run builds locally, exclude local environment folders
like `node_modules`, `.venv`, or `vendor` at all nesting levels using the
`**/` prefix.
### Secrets and Metadata
You should exclude sensitive files and version control metadata to keep your image clean, small, and secure.
You should exclude sensitive files and version control metadata to keep your
image clean, small, and secure.
It's recommend to add `.env`, any encrypted secrets, `.vscode`, `.github`, and anything not required when running in production.
It's recommended to add `.env`, any encrypted secrets, `.vscode`, `.github`, and
anything not required when running in production.
Here's a great [.dockerignore](https://configs.sh/dockerignore/) generator to use as a starting point.
Here's a great [.dockerignore](https://configs.sh/dockerignore/) generator to
use as a starting point.
examples/dockerignore/.dockerignore
+6 -3
View File
@@ -12,8 +12,7 @@
# test glob on hidden files (.env-specific) with a trailing glob
.env*
# should ignore all __pycache__ folders, but not any file named __pycache__
# I've intentionally named a file __pycache__ to test this behavior
# trailing slashes are stripped, so this excludes files and folders named __pycache__
__pycache__/
# test root-level folder
@@ -25,7 +24,7 @@ __pycache__/
# leading glob
*.log
# non-hidden filename, anywhere
# non-hidden filename at the root
Justfile
# root-level file glob (app/TODO should be included)
@@ -43,3 +42,7 @@ negation_test/*
!negation_test/should_not_exist.txt
!negation_test/folder_does_not_exist/
!negation_test/existing_folder/
# test that ** wildcard matches nested directories
# This should exclude both /node_modules and /web/node_modules
**/node_modules
examples/dockerignore/.gitignore
+2
View File
@@ -0,0 +1,2 @@
# keep node_modules to test the inversion rules
!node_modules
examples/dockerignore/node_modules/root-package/index.js
Generated Vendored
+1
View File
@@ -0,0 +1 @@
// Root-level node_modules - should be excluded by "node_modules" pattern
examples/dockerignore/start.sh
+37
View File
@@ -1 +1,38 @@
set -eu
for path in \
app/Justfile \
app/README.md \
app/TODO \
negation_test/should_exist.txt \
negation_test/existing_folder/file_in_folder.txt \
start.sh \
web/app.js
do
if [ ! -e "$path" ]; then
echo "expected included path to exist: $path" >&2
exit 1
fi
done
for path in \
.env-specific \
.envother \
Justfile \
README.md \
__pycache__ \
docker-compose-2.yml \
docker-compose.yml \
node_modules/root-package/index.js \
the.log \
tmp/.should-hide \
tmp/should-hide \
web/node_modules/some-package/index.js
do
if [ -e "$path" ]; then
echo "expected excluded path not to exist: $path" >&2
exit 1
fi
done
find . | sort
examples/dockerignore/test.json
+2 -2
View File
@@ -1,5 +1,5 @@
[
{
"expectedOutput": ["start.sh", "should_exist.txt", "existing_folder/file_in_folder.txt"]
"expectedOutput": ["start.sh", "should_exist.txt", "existing_folder/file_in_folder.txt", "web/app.js"]
}
]
]
examples/dockerignore/web/app.js
+1
View File
@@ -0,0 +1 @@
// This file should be included in the build
examples/dockerignore/web/node_modules/some-package/index.js
Generated Vendored
+1
View File
@@ -0,0 +1 @@
// This file should be excluded by the node_modules pattern
examples/mise-config/.dockerignore
+1
View File
@@ -0,0 +1 @@
# This comment-only file verifies that an empty .dockerignore does not result in an error.
examples/node-vite-react-router-spa/.dockerignore
+4 -1
View File
@@ -1,4 +1,7 @@
.react-router
build
node_modules
README.md
README.md
# not required, but this is to test the .dockerignore parsing logic
!app