Implement embedded HTTPS Git synchronization

This commit is contained in:
Hermes Agent
2026-08-09 23:54:10 +00:00
parent 410007c012
commit 75ce19da00
14 changed files with 4811 additions and 15 deletions

1666
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -18,13 +18,18 @@ cap-std = "4.0"
cap-tempfile = "4.0" cap-tempfile = "4.0"
clap = { version = "4.6", features = ["derive"] } clap = { version = "4.6", features = ["derive"] }
crossterm = "0.29" crossterm = "0.29"
flate2 = "1.1"
gix = { version = "0.86", default-features = false, features = ["blocking-http-transport-reqwest-rust-tls", "index", "merge", "revision", "sha1", "tree-editor"] }
gix-config = "0.59"
iced = "0.14" iced = "0.14"
ironstorage = { path = "crates/storage" } ironstorage = { path = "crates/storage" }
pgp = { version = "0.20", default-features = false } pgp = { version = "0.20", default-features = false }
rand = "0.8" rand = "0.8"
regex = "1.13" regex = "1.13"
reqwest = { version = "0.13", default-features = false, features = ["blocking", "rustls"] }
ratatui = { version = "0.30", default-features = false, features = ["crossterm_0_29", "layout-cache", "macros", "underline-color"] } ratatui = { version = "0.30", default-features = false, features = ["crossterm_0_29", "layout-cache", "macros", "underline-color"] }
serde = { version = "1", features = ["derive"] } serde = { version = "1", features = ["derive"] }
sha1 = "0.10"
shlex = "1.3" shlex = "1.3"
toml = "0.9" toml = "0.9"
uniffi = "0.32" uniffi = "0.32"

View File

@@ -18,6 +18,9 @@ The current direct dependencies are:
| [crossterm 0.29](https://crates.io/crates/crossterm/0.29.0) | Terminal I/O | MIT | | [crossterm 0.29](https://crates.io/crates/crossterm/0.29.0) | Terminal I/O | MIT |
| [Ratatui 0.30](https://crates.io/crates/ratatui/0.30.2) | TUI | MIT | | [Ratatui 0.30](https://crates.io/crates/ratatui/0.30.2) | TUI | MIT |
| [Iced 0.14](https://crates.io/crates/iced/0.14.0) | Desktop UI | MIT | | [Iced 0.14](https://crates.io/crates/iced/0.14.0) | Desktop UI | MIT |
| [gix 0.86](https://crates.io/crates/gix/0.86.0), [gix-config 0.59](https://crates.io/crates/gix-config/0.59.0) | Embedded Git objects, index, references, fetch, and merge | MIT OR Apache-2.0 |
| [reqwest 0.13](https://crates.io/crates/reqwest/0.13.4) | HTTPS smart-Git transport with Rustls | MIT OR Apache-2.0 |
| [flate2 1.1](https://crates.io/crates/flate2/1.1.9), [sha1 0.10](https://crates.io/crates/sha1/0.10.7) | Git pack compression and checksums | MIT OR Apache-2.0 |
| [pgp 0.20](https://crates.io/crates/pgp/0.20.0) | Embedded OpenPGP key import, encryption, decryption, and signatures | MIT OR Apache-2.0 | | [pgp 0.20](https://crates.io/crates/pgp/0.20.0) | Embedded OpenPGP key import, encryption, decryption, and signatures | MIT OR Apache-2.0 |
| [rand 0.8](https://crates.io/crates/rand/0.8.7) | Operating-system-backed cryptographic randomness for OpenPGP operations | MIT OR Apache-2.0 | | [rand 0.8](https://crates.io/crates/rand/0.8.7) | Operating-system-backed cryptographic randomness for OpenPGP operations | MIT OR Apache-2.0 |
| [regex 1.13](https://crates.io/crates/regex/1.13.1) | Linear-time byte-oriented decrypted grep matching | MIT OR Apache-2.0 | | [regex 1.13](https://crates.io/crates/regex/1.13.1) | Linear-time byte-oriented decrypted grep matching | MIT OR Apache-2.0 |
@@ -43,7 +46,7 @@ decision.
| GPG-compatible packets, encryption, and transferable keys | [`pgp` 0.20](https://crates.io/crates/pgp/0.20.0) | MIT OR Apache-2.0 | Selected with default features disabled. The fixture harness proves armored/binary protected key import, packet validation, GPG-compatible decryption, multi-recipient encryption, and detached signatures without native libraries or processes. | | GPG-compatible packets, encryption, and transferable keys | [`pgp` 0.20](https://crates.io/crates/pgp/0.20.0) | MIT OR Apache-2.0 | Selected with default features disabled. The fixture harness proves armored/binary protected key import, packet validation, GPG-compatible decryption, multi-recipient encryption, and detached signatures without native libraries or processes. |
| Alternative GPG implementation | [`sequoia-openpgp` 2.4](https://crates.io/crates/sequoia-openpgp/2.4.1) | LGPL-2.0-or-later | Hold in reserve. Its default Nettle backend is native; its Rust backend exists, but the LGPL adds distribution work we can avoid. | | Alternative GPG implementation | [`sequoia-openpgp` 2.4](https://crates.io/crates/sequoia-openpgp/2.4.1) | LGPL-2.0-or-later | Hold in reserve. Its default Nettle backend is native; its Rust backend exists, but the LGPL adds distribution work we can avoid. |
| GnuPG integration | [`gpgme` 0.11](https://crates.io/crates/gpgme/0.11.0) | LGPL-2.1 | Reject: native GPGME/GnuPG integration and GPG engine processes violate the portability and no-process requirements. | | GnuPG integration | [`gpgme` 0.11](https://crates.io/crates/gpgme/0.11.0) | LGPL-2.1 | Reject: native GPGME/GnuPG integration and GPG engine processes violate the portability and no-process requirements. |
| Local Git plus HTTPS fetch/push | [`gix` 0.86](https://crates.io/crates/gix/0.86.0) | MIT OR Apache-2.0 | Preferred with default features off and `blocking-http-transport-reqwest-rust-tls`; accept HTTPS remotes only and supply credentials directly. | | Local Git plus HTTPS fetch/push | [`gix` 0.86](https://crates.io/crates/gix/0.86.0) | MIT OR Apache-2.0 | Selected with default features off and `blocking-http-transport-reqwest-rust-tls`; accept HTTPS remotes only, supply credentials directly, and use the storage-owned receive-pack implementation for push. |
| Git FFI fallback | [`git2` 0.21](https://crates.io/crates/git2/0.21.0) | MIT OR Apache-2.0 | Reject for now; it links libgit2 and is unnecessary for the HTTPS-only scope. | | Git FFI fallback | [`git2` 0.21](https://crates.io/crates/git2/0.21.0) | MIT OR Apache-2.0 | Reject for now; it links libgit2 and is unnecessary for the HTTPS-only scope. |
| Server/application credentials | [`keyring-core` 1.0](https://crates.io/crates/keyring-core/1.0.0), [`apple-native-keyring-store`](https://crates.io/crates/apple-native-keyring-store/1.0.2), [`windows-native-keyring-store`](https://crates.io/crates/windows-native-keyring-store/1.1.0), [`zbus-secret-service-keyring-store`](https://crates.io/crates/zbus-secret-service-keyring-store/1.0.0) | MIT OR Apache-2.0 | Preferred per-platform stores. The Apple protected store supports iOS/macOS protected data and biometric access. Use the Linux store's Rust crypto feature. | | Server/application credentials | [`keyring-core` 1.0](https://crates.io/crates/keyring-core/1.0.0), [`apple-native-keyring-store`](https://crates.io/crates/apple-native-keyring-store/1.0.2), [`windows-native-keyring-store`](https://crates.io/crates/windows-native-keyring-store/1.1.0), [`zbus-secret-service-keyring-store`](https://crates.io/crates/zbus-secret-service-keyring-store/1.0.0) | MIT OR Apache-2.0 | Preferred per-platform stores. The Apple protected store supports iOS/macOS protected data and biometric access. Use the Linux store's Rust crypto feature. |
| Secret values in memory | [`secrecy` 0.10](https://crates.io/crates/secrecy/0.10.3), [`zeroize` 1.9](https://crates.io/crates/zeroize/1.9.0) | MIT OR Apache-2.0 | `zeroize` selected for the storage-owned redacted byte type; consider `secrecy` only when typed exposure controls add value. | | Secret values in memory | [`secrecy` 0.10](https://crates.io/crates/secrecy/0.10.3), [`zeroize` 1.9](https://crates.io/crates/zeroize/1.9.0) | MIT OR Apache-2.0 | `zeroize` selected for the storage-owned redacted byte type; consider `secrecy` only when typed exposure controls add value. |

View File

@@ -26,6 +26,8 @@ Candidate libraries and the pending project-license decision are tracked in
The shared TOML schema, path rules, editor precedence, and HTTPS remote format The shared TOML schema, path rules, editor precedence, and HTTPS remote format
are documented in [`docs/configuration.md`](docs/configuration.md). are documented in [`docs/configuration.md`](docs/configuration.md).
Embedded Git, HTTPS synchronization, merge behavior, and commit signing are
documented in [`docs/git-synchronization.md`](docs/git-synchronization.md).
The capability-scoped password-store layout and atomic mutation guarantees are The capability-scoped password-store layout and atomic mutation guarantees are
documented in [`docs/repository-core.md`](docs/repository-core.md). documented in [`docs/repository-core.md`](docs/repository-core.md).
The embedded OpenPGP backend, exported-key model, secret-provider boundary, and The embedded OpenPGP backend, exported-key model, secret-provider boundary, and

View File

@@ -10,21 +10,24 @@ publish = false
cap-std.workspace = true cap-std.workspace = true
cap-tempfile.workspace = true cap-tempfile.workspace = true
clap.workspace = true clap.workspace = true
flate2.workspace = true
gix.workspace = true
gix-config.workspace = true
pgp.workspace = true pgp.workspace = true
rand.workspace = true rand.workspace = true
regex.workspace = true regex.workspace = true
reqwest.workspace = true
serde.workspace = true serde.workspace = true
sha1.workspace = true
shlex.workspace = true shlex.workspace = true
toml.workspace = true toml.workspace = true
url.workspace = true url.workspace = true
zeroize.workspace = true zeroize.workspace = true
[dev-dependencies] [dev-dependencies]
flate2 = "1.1"
hex = "0.4" hex = "0.4"
rand_chacha = "0.3" rand_chacha = "0.3"
rustix = { version = "1.1", features = ["fs"] } rustix = { version = "1.1", features = ["fs"] }
sha1 = "0.10"
sha2 = "0.10" sha2 = "0.10"
smallvec = "1.15" smallvec = "1.15"
tempfile = "3" tempfile = "3"

View File

@@ -11,8 +11,8 @@ use std::{
use cap_std::{ambient_authority, fs::Dir}; use cap_std::{ambient_authority, fs::Dir};
use pgp::{ use pgp::{
composed::{ composed::{
Deserializable, DetachedSignature, Esk, Message, MessageBuilder, PublicOrSecret, ArmorOptions, Deserializable, DetachedSignature, Esk, Message, MessageBuilder,
SignedPublicKey, SignedPublicSubKey, SignedSecretKey, SubpacketConfig, PublicOrSecret, SignedPublicKey, SignedPublicSubKey, SignedSecretKey, SubpacketConfig,
}, },
crypto::{hash::HashAlgorithm, sym::SymmetricKeyAlgorithm}, crypto::{hash::HashAlgorithm, sym::SymmetricKeyAlgorithm},
packet::{SignatureType, Subpacket, SubpacketData}, packet::{SignatureType, Subpacket, SubpacketData},
@@ -492,6 +492,18 @@ impl KeyStore {
Ok(DetachedSignatureBytes(bytes)) Ok(DetachedSignatureBytes(bytes))
} }
/// Encode a detached signature as the ASCII armor required by Git's
/// `gpgsig` commit header.
pub fn armor_signature(
&self,
signature: &DetachedSignatureBytes,
) -> Result<Vec<u8>, CryptoError> {
DetachedSignature::from_bytes(Cursor::new(signature.as_bytes()))
.map_err(|_| CryptoError::SigningFailed)?
.to_armored_bytes(ArmorOptions::default())
.map_err(|_| CryptoError::SigningFailed)
}
/// Verify a detached `.gpg-id.sig` against an explicit set of allowed primary identities. /// Verify a detached `.gpg-id.sig` against an explicit set of allowed primary identities.
pub fn verify( pub fn verify(
&self, &self,

2464
crates/storage/src/git.rs Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -9,6 +9,7 @@ pub mod command;
pub mod config; pub mod config;
pub mod crypto; pub mod crypto;
pub mod generate; pub mod generate;
pub mod git;
pub mod mutation; pub mod mutation;
pub mod read; pub mod read;
pub mod recipient; pub mod recipient;

View File

@@ -24,6 +24,7 @@ pub struct TreeCommit {
action: MutationAction, action: MutationAction,
source: String, source: String,
destination: Option<String>, destination: Option<String>,
changed_paths: Vec<std::path::PathBuf>,
message: String, message: String,
} }
@@ -37,6 +38,9 @@ impl TreeCommit {
pub fn destination(&self) -> Option<&str> { pub fn destination(&self) -> Option<&str> {
self.destination.as_deref() self.destination.as_deref()
} }
pub fn changed_paths(&self) -> &[std::path::PathBuf] {
&self.changed_paths
}
pub fn message(&self) -> &str { pub fn message(&self) -> &str {
&self.message &self.message
} }
@@ -148,6 +152,16 @@ impl<'a> TreeMutator<'a> {
action: MutationAction::Remove, action: MutationAction::Remove,
source: display.clone(), source: display.clone(),
destination: None, destination: None,
changed_paths: entries
.iter()
.map(|entry| entry.source.encrypted_relative_path())
.chain(policies.iter().flat_map(|policy| {
[
policy.directory.as_path().join(".gpg-id"),
policy.directory.as_path().join(".gpg-id.sig"),
]
}))
.collect(),
message: format!("Remove {display} from store."), message: format!("Remove {display} from store."),
}; };
if let Err(error) = committer.commit(&change) { if let Err(error) = committer.commit(&change) {
@@ -452,10 +466,34 @@ impl<'a> TreeMutator<'a> {
MutationAction::Copy MutationAction::Copy
}; };
let verb = if moving { "Rename" } else { "Copy" }; let verb = if moving { "Rename" } else { "Copy" };
let mut changed_paths = entries
.iter()
.map(|entry| entry.destination.encrypted_relative_path())
.chain(policies.iter().flat_map(|policy| {
[
policy.destination.as_path().join(".gpg-id"),
policy.destination.as_path().join(".gpg-id.sig"),
]
}))
.collect::<Vec<_>>();
if moving {
changed_paths.extend(
source_entries
.iter()
.map(|entry| entry.source.encrypted_relative_path()),
);
changed_paths.extend(source_policies.iter().flat_map(|policy| {
[
policy.directory.as_path().join(".gpg-id"),
policy.directory.as_path().join(".gpg-id.sig"),
]
}));
}
let change = TreeCommit { let change = TreeCommit {
action, action,
source: source.to_owned(), source: source.to_owned(),
destination: Some(destination.to_owned()), destination: Some(destination.to_owned()),
changed_paths,
message: format!("{verb} {source} to {destination}."), message: format!("{verb} {source} to {destination}."),
}; };
if let Err(error) = committer.commit(&change) { if let Err(error) = committer.commit(&change) {

View File

@@ -45,7 +45,7 @@ impl EntryPath {
DirectoryPath(self.0.parent().map_or_else(PathBuf::new, Path::to_path_buf)) DirectoryPath(self.0.parent().map_or_else(PathBuf::new, Path::to_path_buf))
} }
fn encrypted_relative_path(&self) -> PathBuf { pub fn encrypted_relative_path(&self) -> PathBuf {
let mut path = self.0.clone(); let mut path = self.0.clone();
let mut file_name = path let mut file_name = path
.file_name() .file_name()

View File

@@ -0,0 +1,558 @@
#![forbid(unsafe_code)]
mod support;
use std::{error::Error, fs, io::Cursor, path::Path, sync::Mutex};
use ironstorage::{
command::{InsertInput, InsertRequest},
config::{Config, ConfigLoader, GitRemote},
crypto::{DetachedSignatureBytes, KeyInfo, KeyStore, SecretProvider, SecretProviderError},
git::{
GitChangeKind, GitCredential, GitCredentialProvider, GitError, GitFetchTransport,
GitIdentity, GitRepository, GitSmartHttpTransport, PullOutcome,
},
repository::{Repository, SecretBytes},
write::{InsertContent, OverwriteDecision, VaultWriter},
};
use pgp::{
composed::{Deserializable as _, DetachedSignature},
ser::Serialize as _,
};
use sha1::Digest as _;
use support::compatibility::FixtureSet;
type TestResult = Result<(), Box<dyn Error>>;
fn identity() -> GitIdentity {
GitIdentity::new("IronStorage Test", "test@ironstorage.invalid").expect("valid identity")
}
fn remote_config(temporary: &tempfile::TempDir) -> Result<Config, Box<dyn Error>> {
fs::create_dir_all(temporary.path().join("config/keys"))?;
fs::create_dir_all(temporary.path().join("native"))?;
fs::create_dir_all(temporary.path().join("vault"))?;
let config_path = temporary.path().join("config/config.toml");
fs::write(
&config_path,
format!(
"vault = {:?}\ndefault_key = \"0123456789ABCDEF0123456789ABCDEF01234567\"\nkey_material = \"keys\"\n[[git.remotes]]\nname = \"origin\"\nurl = \"https://example.test/store.git\"\nserver_id = \"server\"\napplication_id = \"application\"\n",
temporary.path().join("vault")
),
)?;
Ok(
ConfigLoader::new(temporary.path().to_owned(), temporary.path().join("native"))
.load(Some(&config_path))?,
)
}
#[test]
fn local_git_workflow_stages_commits_diffs_logs_and_deletes() -> TestResult {
let temporary = tempfile::tempdir()?;
let store = Repository::open(temporary.path())?;
let git = GitRepository::init(&store, identity())?;
fs::write(temporary.path().join(".gpg-id"), b"ALICE\n")?;
git.stage(&[".gpg-id".into()])?;
let first = git.commit("Set password store recipients.")?;
assert_eq!(first.len(), 40);
assert!(git.status()?.is_clean());
fs::write(temporary.path().join(".gpg-id"), b"BOB\n")?;
let status = git.status()?;
assert_eq!(status.unstaged().len(), 1);
assert_eq!(status.unstaged()[0].kind(), GitChangeKind::Modified);
let diff = git.diff(&[])?;
assert_eq!(diff[0].old(), Some(b"ALICE\n".as_slice()));
assert_eq!(diff[0].current(), Some(b"BOB\n".as_slice()));
git.stage(&[".gpg-id".into()])?;
assert_eq!(git.status()?.staged()[0].kind(), GitChangeKind::Modified);
git.commit("Change password store recipients.")?;
assert_eq!(git.log(None)?.len(), 3);
assert_eq!(
git.log(Some(1))?[0].message(),
"Change password store recipients."
);
assert_eq!(git.stage_and_commit(&[".gpg-id".into()], "No-op")?, None);
fs::remove_file(temporary.path().join(".gpg-id"))?;
git.stage(&[".gpg-id".into()])?;
let deletion = git.status()?;
assert_eq!(deletion.staged().len(), 1, "{deletion:?}");
assert_eq!(deletion.staged()[0].kind(), GitChangeKind::Deleted);
git.commit("Remove password store recipients.")?;
assert!(git.status()?.is_clean());
Ok(())
}
#[test]
fn nested_repository_selection_is_innermost() -> TestResult {
let temporary = tempfile::tempdir()?;
let outer_store = Repository::open(temporary.path())?;
GitRepository::init(&outer_store, identity())?;
fs::create_dir(temporary.path().join("nested"))?;
let inner_store = Repository::open(temporary.path().join("nested"))?;
GitRepository::init(&inner_store, identity())?;
fs::write(temporary.path().join("nested/secret.gpg"), b"ciphertext")?;
let selected =
GitRepository::open_innermost(&outer_store, Path::new("nested/secret.gpg"), identity())?;
assert_eq!(selected.root(), temporary.path().join("nested"));
Ok(())
}
#[test]
fn remotes_and_config_are_local_https_only() -> TestResult {
let temporary = tempfile::tempdir()?;
let store = Repository::open(temporary.path())?;
let mut git = GitRepository::init(&store, identity())?;
for forbidden in [
"ssh://example.test/store.git",
"git@example.test:store.git",
"git://example.test/store.git",
"file:///tmp/store.git",
"../store.git",
"ext::helper command",
"https://user:secret@example.test/store.git",
] {
assert_eq!(
git.add_remote("origin", forbidden),
Err(GitError::ForbiddenRemoteUrl)
);
}
git.add_remote("origin", "https://example.test/store.git")?;
assert_eq!(git.remotes(), ["origin"]);
assert_eq!(git.remote_url("origin")?, "https://example.test/store.git");
git.set_remote_url("origin", "https://example.test/other.git")?;
assert_eq!(git.remote_url("origin")?, "https://example.test/other.git");
git.config_set("user.name", "Local User")?;
assert_eq!(git.config_get("user.name")?.as_deref(), Some("Local User"));
assert!(git.config_set("credential.helper", "evil").is_err());
git.remove_remote("origin")?;
assert!(git.remotes().is_empty());
Ok(())
}
struct Credentials;
impl GitCredentialProvider for Credentials {
fn credential(
&self,
server: &ironstorage::config::ServerId,
application: &ironstorage::config::ApplicationId,
) -> Result<GitCredential, GitError> {
assert_eq!(server.as_str(), "server");
assert_eq!(application.as_str(), "application");
GitCredential::new("alice", b"token".to_vec())
}
}
struct SigningSecret(Vec<u8>);
impl SecretProvider for SigningSecret {
fn secret_for(&mut self, _key: &KeyInfo) -> Result<SecretBytes, SecretProviderError> {
Ok(SecretBytes::new(self.0.clone()))
}
}
#[derive(Default)]
struct RecordingTransport {
request: Mutex<Vec<u8>>,
}
struct CloningFetch;
impl GitFetchTransport for CloningFetch {
fn fetch(
&self,
repository: &GitRepository,
_configured: &GitRemote,
credential: &GitCredential,
) -> Result<bool, GitError> {
assert_eq!(credential.password(), b"token");
fs::write(repository.root().join(".gpg-id"), b"ALICE\n")
.map_err(|error| GitError::InvalidRepository(error.to_string()))?;
repository.stage(&[".gpg-id".into()])?;
let tip = repository.commit("Fetched initial store.")?;
set_remote_tracking(repository.root(), &tip)
.map_err(|error| GitError::InvalidRepository(error.to_string()))?;
reset_to_unborn(repository.root())
.map_err(|error| GitError::InvalidRepository(error.to_string()))?;
Ok(true)
}
}
struct NoopFetch;
impl GitFetchTransport for NoopFetch {
fn fetch(
&self,
_repository: &GitRepository,
_configured: &GitRemote,
credential: &GitCredential,
) -> Result<bool, GitError> {
assert_eq!(credential.username(), "alice");
Ok(false)
}
}
struct AuthenticationFailure;
impl GitSmartHttpTransport for AuthenticationFailure {
fn advertise_receive_pack(
&self,
_url: &url::Url,
_credential: &GitCredential,
) -> Result<Vec<u8>, GitError> {
Err(GitError::AuthenticationFailed)
}
fn receive_pack(
&self,
_url: &url::Url,
_credential: &GitCredential,
_request: Vec<u8>,
) -> Result<Vec<u8>, GitError> {
unreachable!("authentication fails before receive-pack")
}
}
struct NonFastForwardAdvertisement;
impl GitSmartHttpTransport for NonFastForwardAdvertisement {
fn advertise_receive_pack(
&self,
_url: &url::Url,
_credential: &GitCredential,
) -> Result<Vec<u8>, GitError> {
let mut output = packet(b"# service=git-receive-pack\n");
output.extend_from_slice(b"0000");
output.extend_from_slice(&packet(
b"1111111111111111111111111111111111111111 refs/heads/main\0report-status\n",
));
output.extend_from_slice(b"0000");
Ok(output)
}
fn receive_pack(
&self,
_url: &url::Url,
_credential: &GitCredential,
_request: Vec<u8>,
) -> Result<Vec<u8>, GitError> {
unreachable!("non-fast-forward is rejected before receive-pack")
}
}
impl GitSmartHttpTransport for RecordingTransport {
fn advertise_receive_pack(
&self,
_url: &url::Url,
credential: &GitCredential,
) -> Result<Vec<u8>, GitError> {
assert_eq!(credential.username(), "alice");
let mut output = packet(b"# service=git-receive-pack\n");
output.extend_from_slice(b"0000");
output.extend_from_slice(&packet(b"0000000000000000000000000000000000000000 capabilities^{}\0report-status delete-refs\n"));
output.extend_from_slice(b"0000");
Ok(output)
}
fn receive_pack(
&self,
_url: &url::Url,
_credential: &GitCredential,
request: Vec<u8>,
) -> Result<Vec<u8>, GitError> {
*self.request.lock().expect("request lock") = request;
let mut output = packet(b"unpack ok\n");
output.extend_from_slice(&packet(b"ok refs/heads/main\n"));
output.extend_from_slice(b"0000");
Ok(output)
}
}
#[test]
fn injected_smart_http_push_sends_a_complete_pack_and_credentials() -> TestResult {
let temporary = tempfile::tempdir()?;
let config = remote_config(&temporary)?;
let remote: &GitRemote = &config.git_remotes()[0];
let store = Repository::open(config.vault())?;
let mut git = GitRepository::init(&store, identity())?;
git.add_remote("origin", remote.url().as_str())?;
fs::write(config.vault().join("secret.gpg"), b"ciphertext")?;
git.stage(&["secret.gpg".into()])?;
let head = git.commit("Add secret to store.")?;
let transport = RecordingTransport::default();
let outcome = git.push_with_transport(remote, Some("main"), &Credentials, &transport)?;
assert_eq!(outcome.new_id(), head);
let request = transport.request.lock().expect("request lock");
let pack_offset = request
.windows(4)
.position(|window| window == b"PACK")
.expect("pack payload");
assert!(
request[..pack_offset]
.windows(b"refs/heads/main".len())
.any(|window| window == b"refs/heads/main")
);
let pack = &request[pack_offset..];
assert_eq!(&pack[..4], b"PACK");
assert_eq!(u32::from_be_bytes(pack[4..8].try_into()?), 2);
assert!(u32::from_be_bytes(pack[8..12].try_into()?) >= 3);
let digest = sha1::Sha1::digest(&pack[..pack.len() - 20]);
assert_eq!(digest.as_slice(), &pack[pack.len() - 20..]);
Ok(())
}
#[test]
fn push_propagates_authentication_and_rejects_non_fast_forward_before_upload() -> TestResult {
let temporary = tempfile::tempdir()?;
let config = remote_config(&temporary)?;
let remote = &config.git_remotes()[0];
let store = Repository::open(config.vault())?;
let mut git = GitRepository::init(&store, identity())?;
git.add_remote("origin", remote.url().as_str())?;
assert_eq!(
git.push_with_transport(remote, Some("main"), &Credentials, &AuthenticationFailure),
Err(GitError::AuthenticationFailed)
);
assert_eq!(
git.push_with_transport(
remote,
Some("main"),
&Credentials,
&NonFastForwardAdvertisement,
),
Err(GitError::NonFastForward)
);
Ok(())
}
#[test]
fn fetched_branches_fast_forward_and_report_typed_conflicts() -> TestResult {
let temporary = tempfile::tempdir()?;
let config = remote_config(&temporary)?;
let remote = &config.git_remotes()[0];
let store = Repository::open(config.vault())?;
let mut git = GitRepository::init(&store, identity())?;
git.add_remote("origin", remote.url().as_str())?;
fs::write(config.vault().join("secret.gpg"), b"base")?;
git.stage(&["secret.gpg".into()])?;
let base = git.commit("Base")?;
fs::write(config.vault().join("secret.gpg"), b"remote")?;
git.stage(&["secret.gpg".into()])?;
let remote_tip = git.commit("Remote change")?;
reset_head_and_index(&git, config.vault(), &base, b"base")?;
set_remote_tracking(config.vault(), &remote_tip)?;
assert_eq!(
git.pull_with_transport(remote, Some("main"), &Credentials, &NoopFetch)?,
PullOutcome::FastForward
);
assert_eq!(fs::read(config.vault().join("secret.gpg"))?, b"remote");
reset_head_and_index(&git, config.vault(), &base, b"base")?;
fs::write(config.vault().join("secret.gpg"), b"local")?;
git.stage(&["secret.gpg".into()])?;
git.commit("Local change")?;
let error = git
.pull_with_transport(remote, Some("main"), &Credentials, &NoopFetch)
.expect_err("conflicting histories");
assert_eq!(
error,
GitError::MergeConflicts {
paths: vec!["secret.gpg".into()]
}
);
assert_eq!(fs::read(config.vault().join("secret.gpg"))?, b"local");
Ok(())
}
#[test]
fn clone_uses_a_private_directory_and_injected_fetch_transport() -> TestResult {
let temporary = tempfile::tempdir()?;
let config = remote_config(&temporary)?;
let destination = temporary.path().join("cloned-vault");
let cloned = GitRepository::clone_into_with_transport(
&destination,
identity(),
&config.git_remotes()[0],
&Credentials,
&CloningFetch,
)?;
assert_eq!(cloned.root(), destination);
assert_eq!(fs::read(destination.join(".gpg-id"))?, b"ALICE\n");
assert_eq!(cloned.remotes(), ["origin"]);
assert_eq!(cloned.log(None)?.len(), 1);
assert_eq!(cloned.log(None)?[0].message(), "Fetched initial store.");
assert!(!destination.join(".gitattributes").exists());
assert!(
!temporary
.path()
.read_dir()?
.filter_map(Result::ok)
.any(|entry| entry
.file_name()
.to_string_lossy()
.starts_with(".ironstorage-clone-"))
);
Ok(())
}
#[test]
fn signed_commits_have_a_verifiable_ascii_armored_gpgsig() -> TestResult {
let fixture = FixtureSet::load()?;
let alice = fixture.key("alice")?;
let keys = KeyStore::load(fixture.path("keys"))?;
let signer = keys.resolve(&alice.primary_fingerprint)?;
let temporary = tempfile::tempdir()?;
let store = Repository::open(temporary.path())?;
let git = GitRepository::init(&store, identity())?;
fs::write(temporary.path().join("secret.gpg"), b"ciphertext")?;
git.stage(&["secret.gpg".into()])?;
git.commit_signed(
"Add signed password.",
&keys,
&signer,
&mut SigningSecret(alice.passphrase.as_bytes().to_vec()),
)?;
let repository = open_test_git(temporary.path())?;
let head = repository.head_commit()?;
let decoded = head.decode()?;
let mut commit = decoded.into_owned()?;
let signature_index = commit
.extra_headers
.iter()
.position(|(name, _)| name.as_slice() == b"gpgsig")
.expect("gpgsig header");
let armor = commit.extra_headers.remove(signature_index).1;
assert!(armor.starts_with(b"-----BEGIN PGP SIGNATURE-----"));
let (signature, _) = DetachedSignature::from_armor_single(Cursor::new(armor))?;
let mut signature_bytes = Vec::new();
signature.to_writer(&mut signature_bytes)?;
let mut unsigned = Vec::new();
gix::objs::WriteTo::write_to(&commit, &mut unsigned)?;
assert_eq!(
keys.verify(
&unsigned,
&DetachedSignatureBytes::new(signature_bytes),
std::slice::from_ref(&signer),
)?,
signer
);
Ok(())
}
#[test]
fn successful_storage_mutations_use_the_embedded_committer() -> TestResult {
let fixture = FixtureSet::load()?;
let store = fixture.materialize_store("basic")?;
let repository = Repository::open(store.path())?;
let keys = KeyStore::load(fixture.path("keys"))?;
let mut git = GitRepository::init(&repository, identity())?;
assert_eq!(
fs::read(store.path().join(".gitattributes"))?,
b"*.gpg diff=gpg\n"
);
let initialized = git.log(None)?;
assert_eq!(initialized.len(), 2);
assert_eq!(
initialized[0].message(),
"Configure git repository for gpg file diff."
);
assert_eq!(
initialized[1].message(),
"Add current contents of password store."
);
fs::write(store.path().join("unrelated.gpg"), b"separately staged")?;
git.stage(&["unrelated.gpg".into()])?;
let writer = VaultWriter::new(&repository, &keys);
writer.insert(
&InsertRequest {
entry: "automatic/entry".to_owned(),
input: InsertInput::EchoedLine,
force: false,
},
InsertContent::echoed(b"generated secret".to_vec())?,
OverwriteDecision::Decline,
None,
&mut git,
)?;
let log = git.log(None)?;
assert_eq!(log.len(), 3);
assert_eq!(
log[0].message(),
"Add given password for automatic/entry to store."
);
let embedded = open_test_git(store.path())?;
let head = embedded.head_commit()?;
let committed = head.tree()?;
assert!(
committed
.lookup_entry_by_path("automatic/entry.gpg")?
.is_some()
);
assert!(committed.lookup_entry_by_path("unrelated.gpg")?.is_none());
let status = git.status()?;
assert_eq!(status.staged().len(), 1, "{status:?}");
assert_eq!(status.staged()[0].path(), Path::new("unrelated.gpg"));
assert_eq!(status.staged()[0].kind(), GitChangeKind::Added);
Ok(())
}
fn reset_head_and_index(git: &GitRepository, root: &Path, id: &str, contents: &[u8]) -> TestResult {
let repository = open_test_git(root)?;
let id = gix::hash::ObjectId::from_hex(id.as_bytes())?;
repository
.head_ref()?
.expect("born HEAD")
.set_target_id(id, "test reset")?;
fs::write(root.join("secret.gpg"), contents)?;
git.stage(&["secret.gpg".into()])?;
Ok(())
}
fn set_remote_tracking(root: &Path, id: &str) -> TestResult {
let repository = open_test_git(root)?;
let id = gix::hash::ObjectId::from_hex(id.as_bytes())?;
if let Some(existing) = repository.try_find_reference("refs/remotes/origin/main")? {
existing.delete()?;
}
repository.reference(
"refs/remotes/origin/main",
id,
gix::refs::transaction::PreviousValue::MustNotExist,
"test remote update",
)?;
Ok(())
}
fn reset_to_unborn(root: &Path) -> TestResult {
let repository = open_test_git(root)?;
repository.find_reference("refs/heads/main")?.delete()?;
let mut index = repository.index_from_tree(&repository.empty_tree().id)?;
index.write(Default::default())?;
fs::remove_file(root.join(".gpg-id"))?;
Ok(())
}
fn open_test_git(root: &Path) -> Result<gix::Repository, Box<dyn Error>> {
Ok(gix::open_opts(
root,
gix::open::Options::isolated().config_overrides([
"user.name=Test".to_owned(),
"user.email=test@ironstorage.invalid".to_owned(),
]),
)?)
}
fn packet(data: &[u8]) -> Vec<u8> {
let mut output = format!("{:04x}", data.len() + 4).into_bytes();
output.extend_from_slice(data);
output
}

View File

@@ -151,6 +151,10 @@ fn copy_preserves_source_and_reencrypts_for_destination_policy() -> TestResult {
committer.changes[0].message(), committer.changes[0].message(),
"Copy email/personal to team/personal." "Copy email/personal to team/personal."
); );
assert_eq!(
committer.changes[0].changed_paths(),
&[Path::new("team/personal.gpg").to_owned()]
);
Ok(()) Ok(())
} }
@@ -365,6 +369,13 @@ fn move_writes_durable_destination_before_removing_source() -> TestResult {
committer.changes[0].message(), committer.changes[0].message(),
"Rename email/personal to archive/personal." "Rename email/personal to archive/personal."
); );
assert_eq!(
committer.changes[0].changed_paths(),
&[
Path::new("archive/personal.gpg").to_owned(),
Path::new("email/personal.gpg").to_owned(),
]
);
Ok(()) Ok(())
} }

View File

@@ -0,0 +1,47 @@
# Embedded Git and synchronization
All Git behavior is implemented in `crates/storage`. IronStorage never launches
`git`, a credential helper, an SSH client, a hook, a filter, or a merge driver.
Repositories are opened with isolated configuration and environment access;
repository-local configuration that could name an executable is rejected.
`GitRepository` initializes and opens password-store worktrees, selects the
innermost repository for a nested entry, and implements status, log, diff, add,
commit, remote, and local-config operations. Successful insert, edit, generate,
recipient-policy, remove, move, and copy transactions use the same concrete
committer. Only their affected paths are staged, unrelated index state is
preserved, no-op mutations create no commit, and commit failures restore the
index so the storage transaction can roll back its files.
## HTTPS transport
Remote URLs must be absolute, credential-free HTTPS URLs. SSH, scp syntax,
`git://`, `file://`, local paths, helper transports, URL rewrites, separate push
URLs, and unknown schemes are rejected before transport. Credentials are
requested with the configured server ID and application ID and remain outside
Git configuration.
Fetch uses the embedded Rust smart-HTTP client with an explicit credential
callback, so Git's credential cascade is never entered. Push implements the
receive-pack protocol directly: it validates the advertisement, checks the
remote tip is an ancestor, creates a complete Git pack with a SHA-1 trailer,
requests `report-status`, and accepts the update only after both unpack and ref
status succeed. HTTP redirects are disabled so authorization cannot cross an
origin boundary.
Pull refuses a dirty worktree. It fast-forwards when possible and otherwise
uses the embedded three-way tree merge. Unresolved paths are returned as typed
`MergeConflicts`; no conflict markers or partial checkout are written. Checkout
prevalidates tree entries, rejects links and submodules, writes private files
atomically, updates the real Git index, and rolls the worktree/index back if the
reference update fails. `sync` performs pull before push, while clone builds in
a private sibling directory and installs the completed vault with a rename.
Commit signing is optional. The embedded OpenPGP key store signs the canonical
unsigned commit bytes and adds an ASCII-armored `gpgsig` header compatible with
Git/GPG without invoking `gpg`.
The smart-HTTP boundary is injectable for deterministic compatibility tests.
Tests can inspect credentials, advertisements, receive-pack commands, object
counts, pack checksums, non-fast-forward behavior, and server status without a
runtime helper or external Git installation.

View File

@@ -53,5 +53,5 @@ no-op, and must restore any staging state before returning an error. The policy
manager then restores repository bytes. `NoGitCommitter` represents a path not manager then restores repository bytes. `NoGitCommitter` represents a path not
contained in a Git work tree; it is not used for a discovered repository. contained in a Git work tree; it is not used for a discovered repository.
The later embedded-Git implementation owns concrete staging and commits, while The embedded Git implementation owns concrete staging and commits, while this
this module owns the all-or-nothing storage mutation contract it invokes. module owns the all-or-nothing storage mutation contract it invokes.