Files
Zach LeslieandGitHub a9f63d073b Replace TenantIndex, BlockMeta and CompactedBlockMeta with proto near-equivalents (#4072)
* Update proto for v1 backend

* Functioning blocklist with new package

* Functioning blocklist with existing package

* Generate proto to existing package

* Add wrapper around google UUID for proto marshaling

* Update blocklist for wrapper type

* Test WriteTenantIndex

* Implement json for uuid

* Almost functional backend tests

* Passing backend tests

* Improve UUID coverage and utility

* TempoDB updates for UUID and struct type changes

* Update method style for uuid.New()

* Update method style for uuid.MustParse()

* Add uuid.From helper

* Update modules for UUID and proto struct types

* Update poller integration test for UUID

* Update tempo-cli for UUID and proto struct type changes

* Lint and clean up

* Fix serverless

* Fix cli

* Fix cli

* Add fixture tests for roundtrip assurance between versions

* Clean up notes from proto definition

* Update backend/test

* Be more specific about unmarshal UUID

* Include event for compacted block

* Use better json tag for compacted meta

* Implement UnmarshalJson on CompcatedBlockMeta for embedded struct

* Test for CompactedMeta marshal/unmarshal

* Drop benchmark index

* Clean commented code

* Extend mocks to capture multiple writes

* Update backend/test fixtures

* Write both proto and json to the backend for all three files in all backends

* Fix lint

* Back UUID with google.UUID to avoid struct

* Relocate pkg/uuid to tempodb/backend

* Re-plumb UUID through tempodb

* Re-plumb UUID through cmd

* Re-plumb UUID through modules

* Re-plumb UUID through integration/

* Drop package spec from tempo.proto

* Improve encoding marshal performance

* Drop block proto and tenantindex json

* Drop additional proto include

* Implement zstd tenant index compression

* Add note about use of json in the dedicatedcolumns

* Fix ingester local block

* Spell

* Revert mock update

* Drop json fallback from tenantindex read

* Replace single-tenant test fixtures

* Update fixture replacement test

* Improve error handling and fix test

* Extend test for tenantindex round-trip

* Small fixes

* Revert "Drop json fallback from tenantindex read"

This reverts commit 834ff5b85d8a8a4d7eb560fbb67ba8f3a5a3959a.

* Add back mocks to capture multiple backend writes

* Add back proto & json writitng/reading for tenantindex

* Change TotalObjects type from int32 -> int64

The original change was int -> int32, which on 64bit machines is a
downgrade that I overlooked.  Raising here to int64 to maintain the
size.

* Store write buffer at object path in mocks to improve readability in tests

* Avoid allocation in interface validation

* Update note to include intern

* Pluralize metas and compacted_metas fields

* Fallback to json only on ErrDoesNotExist

* Lint

* Drop debug

* Revert "Pluralize metas and compacted_metas fields"

This reverts commit 46a8c36e37ac33e1ffc104287d898930e02cdd97.

* Import original tenantindex and test contents for backwards compatibility

* Update changelog
2024-10-02 18:40:59 +00:00

47 lines
931 B
Go

package vparquet3
import (
"context"
"testing"
"github.com/stretchr/testify/require"
"github.com/grafana/tempo/tempodb/backend"
"github.com/grafana/tempo/tempodb/backend/local"
)
func TestRawIteratorReadsAllRows(t *testing.T) {
rawR, _, _, err := local.New(&local.Config{
Path: "./test-data",
})
require.NoError(t, err)
r := backend.NewReader(rawR)
ctx := context.Background()
blocks, _, err := r.Blocks(ctx, "single-tenant")
require.NoError(t, err)
require.Len(t, blocks, 1)
meta, err := r.BlockMeta(ctx, blocks[0], "single-tenant")
require.NoError(t, err)
b := newBackendBlock(meta, r)
iter, err := b.rawIter(context.Background(), newRowPool(10))
require.NoError(t, err)
defer iter.Close()
actualCount := int64(0)
for {
_, tr, err := iter.Next(context.Background())
if tr == nil {
break
}
actualCount++
require.NoError(t, err)
}
require.Equal(t, meta.TotalObjects, actualCount)
}