Files
deploy/mirror/hooks/fsmonitor-watchman.sample
zooqueenandHanzo Dev 649cac9021 mirror: run in a repository, and ship as a subcommand of the one binary
Reconcile could never have worked. RealGit set no working directory, so every
command ran wherever the process started — for a scheduled job, nowhere — and git
answers "fatal: not a git repository" to the fetch. That error arrives on the
branch that reports GITHUB as unreadable, so a reconciler with perfect
credentials and a perfect table would have called all 24 repos unreachable and
asked a person to look at each one. The dry run passed throughout, because a dry
run resolves names and never touches git.

Git therefore takes the directory as a parameter, and Workspace prepares one
store per repo under a root that MAY survive between runs. May, not must: git's
fetch negotiation offers whatever is already present, so a kept root makes later
runs deltas, and an empty one is equally correct and merely slower. That is what
lets the job use an emptyDir instead of a claim — a ReadWriteOnce volume would
buy the same time at the price of a job pod deadlocking on Multi-Attach whenever
it schedules away from the node holding it.

Both sides are now fetched, not just GitHub. The forge goes first for two
reasons: its tip has to be local before a genuine divergence can be NAMED rather
than reported as "ancestry undetermined", and once its history is present those
objects are offered in negotiation, so the internet side of every run is a delta
even on an empty store. That is the whole reason ten-minute scheduling is
affordable over two dozen repositories.

The proof is a real one. TestFastForwardMovesARealRepository builds two bare
repositories, leaves the forge a commit behind, and requires it to end on
GitHub's tip — then rewinds GitHub so the two genuinely disagree and requires the
forge NOT to move. Every other test in this file uses a fake that agrees with the
code about what git is, which is exactly why none of them caught this. Verified
by mutation: drop `cmd.Dir` and it fails.

Shipping is one binary, not two. cmd/hanzo-mirror becomes cmd/cd-mirror,
dispatched by argv[0] like every other cd command, so the reconciler travels in
the image the control plane already builds, pins and deploys — no second artifact
to version, and no second chance for the two to disagree about which repos exist.
The table travels with it at common.DefaultMirrorTable rather than living in a
ConfigMap, so the JSON that CI validates is the JSON the scheduled run reads.

Co-authored-by: Hanzo Dev <[email protected]>
2026-08-04 16:53:25 -07:00

169 lines
4.5 KiB
Perl
Executable File

#!/usr/bin/perl
use strict;
use warnings;
use IPC::Open2;
# An example hook script to integrate Watchman
# (https://facebook.github.io/watchman/) with git to speed up detecting
# new and modified files.
#
# The hook is passed a version (currently 2) and last update token
# formatted as a string and outputs to stdout a new update token and
# all files that have been modified since the update token. Paths must
# be relative to the root of the working tree and separated by a single NUL.
#
# To enable this hook, rename this file to "query-watchman" and set
# 'git config core.fsmonitor .git/hooks/query-watchman'
#
my ($version, $last_update_token) = @ARGV;
# Uncomment for debugging
# print STDERR "$0 $version $last_update_token\n";
# Check the hook interface version
if ($version ne 2) {
die "Unsupported query-fsmonitor hook version '$version'.\n" .
"Falling back to scanning...\n";
}
my $git_work_tree = get_working_dir();
my $json_pkg;
eval {
require JSON::XS;
$json_pkg = "JSON::XS";
1;
} or do {
require JSON::PP;
$json_pkg = "JSON::PP";
};
launch_watchman();
sub launch_watchman {
my $o = watchman_query();
if (is_work_tree_watched($o)) {
output_result($o->{clock}, @{$o->{files}});
}
}
sub output_result {
my ($clockid, @files) = @_;
# Uncomment for debugging watchman output
# open (my $fh, ">", ".git/watchman-output.out");
# binmode $fh, ":utf8";
# print $fh "$clockid\n@files\n";
# close $fh;
binmode STDOUT, ":utf8";
print $clockid;
print "\0";
local $, = "\0";
print @files;
}
sub watchman_clock {
my $response = qx/watchman clock "$git_work_tree"/;
die "Failed to get clock id on '$git_work_tree'.\n" .
"Falling back to scanning...\n" if $? != 0;
return $json_pkg->new->utf8->decode($response);
}
sub watchman_query {
my $pid = open2(\*CHLD_OUT, \*CHLD_IN, 'watchman -j --no-pretty')
or die "open2() failed: $!\n" .
"Falling back to scanning...\n";
# In the query expression below we're asking for names of files that
# changed since $last_update_token but not from the .git folder.
#
# To accomplish this, we're using the "since" generator to use the
# recency index to select candidate nodes and "fields" to limit the
# output to file names only. Then we're using the "expression" term to
# further constrain the results.
my $last_update_line = "";
if (substr($last_update_token, 0, 1) eq "c") {
$last_update_token = "\"$last_update_token\"";
$last_update_line = qq[\n"since": $last_update_token,];
}
my $query = <<" END";
["query", "$git_work_tree", {$last_update_line
"fields": ["name"],
"expression": ["not", ["dirname", ".git"]]
}]
END
# Uncomment for debugging the watchman query
# open (my $fh, ">", ".git/watchman-query.json");
# print $fh $query;
# close $fh;
print CHLD_IN $query;
close CHLD_IN;
my $response = do {local $/; <CHLD_OUT>};
# Uncomment for debugging the watch response
# open ($fh, ">", ".git/watchman-response.json");
# print $fh $response;
# close $fh;
die "Watchman: command returned no output.\n" .
"Falling back to scanning...\n" if $response eq "";
die "Watchman: command returned invalid output: $response\n" .
"Falling back to scanning...\n" unless $response =~ /^\{/;
return $json_pkg->new->utf8->decode($response);
}
sub is_work_tree_watched {
my ($output) = @_;
my $error = $output->{error};
if ($error and $error =~ m/unable to resolve root .* directory (.*) is not watched/) {
my $response = qx/watchman watch "$git_work_tree"/;
die "Failed to make watchman watch '$git_work_tree'.\n" .
"Falling back to scanning...\n" if $? != 0;
$output = $json_pkg->new->utf8->decode($response);
$error = $output->{error};
die "Watchman: $error.\n" .
"Falling back to scanning...\n" if $error;
# Uncomment for debugging watchman output
# open (my $fh, ">", ".git/watchman-output.out");
# close $fh;
# Watchman will always return all files on the first query so
# return the fast "everything is dirty" flag to git and do the
# Watchman query just to get it over with now so we won't pay
# the cost in git to look up each individual file.
my $o = watchman_clock();
$error = $o->{error};
die "Watchman: $error.\n" .
"Falling back to scanning...\n" if $error;
output_result($o->{clock}, ("/"));
return 0;
}
die "Watchman: $error.\n" .
"Falling back to scanning...\n" if $error;
return 1;
}
sub get_working_dir {
my $working_dir;
if ($^O =~ 'msys' || $^O =~ 'cygwin') {
$working_dir = Win32::GetCwd();
$working_dir =~ tr/\\/\//;
} else {
require Cwd;
$working_dir = Cwd::cwd();
}
return $working_dir;
}