Using Pinecone.Client with Testcontainers and the local emulator

[adamsitnik]

Hello!

I am working on updating the Semantic Kernel connector for Pinecone to use the official SDK: microsoft/semantic-kernel#10415

The first step that I tried to take was to setup a test container to use the local emulator so I could write some integration tests.

Here is the code I have:

using DotNet.Testcontainers.Builders;
using Pinecone;

namespace PineconeLocalRepro
{
    internal class Program
    {
        static async Task Main(string[] args)
        {
            // Values taken from https://docs.pinecone.io/guides/operations/local-development
            const string Image = "ghcr.io/pinecone-io/pinecone-local:latest";
            const ushort HttpPort = 5080;
            const string Host = "localhost";

            var container = new ContainerBuilder()
                .WithImage(Image)
                .WithPortBinding(HttpPort, assignRandomHostPort: true)
                .WithEnvironment("PINECONE_HOST", "localhost")
                .WithEnvironment("PORT", HttpPort.ToString())
                .Build();

            await container.StartAsync();

            await Task.Delay(TimeSpan.FromSeconds(3)); // add some artificial delay

            string url = $"http://{container.Hostname}:{container.GetMappedPublicPort(HttpPort)}";
            Console.WriteLine($"Using URL: '{url}'");

            var pc = new PineconeClient("pclocal",
                new ClientOptions
                {
                    BaseUrl = url,
                    IsTlsEnabled = false
                }
            );

            await pc.ListCollectionsAsync();

            var collectionModel = await pc.CreateCollectionAsync(new CreateCollectionRequest
            {
                Name = "example-collection",
                Source = "example-index",
            });
        }
    }
}

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Pinecone.Client" Version="3.0.0" />
    <PackageReference Include="Testcontainers" Version="4.3.0" />
  </ItemGroup>

</Project>

I keeps failing with "The response ended prematurely":

Using URL: 'http://127.0.0.1:63815'
Unhandled exception. System.Net.Http.HttpRequestException: An error occurred while sending the request.
 ---> System.Net.Http.HttpIOException: The response ended prematurely. (ResponseEnded)
   at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken o
   at Pinecone.Core.RawClient.SendWithRetriesAsync(BaseApiRequest request, CancellationToken cancellationToken)
   at Pinecone.Core.RawClient.MakeRequestAsync(BaseApiRequest request, CancellationToken cancellationToken)
   at Pinecone.BasePinecone.ListCollectionsAsync(RequestOptions options, CancellationToken cancellationToken)
   at PineconeLocalRepro.Program.Main(String[] args) in C:\Users\adsitnik\source\repos\PineconeLocalRepro\Program.cs:line 37
   at PineconeLocalRepro.Program.<Main>(String[] args)

am I doing something wrong?

[adamsitnik]

cc @fern-support

[eddumelendez]

I think this is similar to the issue I have with java sdk. I reported it here.

I also would like to use it with Testcontainers :)

[dannysheridan]

@adamsitnik acknowledging receipt from fern. thanks for the thorough issue.

[adamsitnik]

I've done some debugging and was able to find the reason why it does not work. First of all, the emulator seems to be outdated and does not work with 3.0 of the client (the client reports that some required JSON fields are missing), but we can get it working with 2.1.

When "host" argument is not provided for PineconeClient.Index, it will try to get the host from the Pinecone service:

pinecone-dotnet-client/src/Pinecone/PineconeClient.cs

Line 30 in f2f39c9

host ??= DescribeIndexAsync(indexName: name!).Result.Host;

In the cloud environment it's fine, but with the local emulator it reports the address with the container port, not the host port. The PINECONE_HOST allows to control the host name, but not the host port. Even if it would, it would not be enough as we are using multiple ports.

What I ended up doing was implementing a RedirectHandler that redirects all the web requests made by the client to the host ports. Which BTW proved that exposing HttpClient as part of the option bag was great, as I did not need any changes in the client itself.

using DotNet.Testcontainers.Builders;
using Pinecone;

namespace PineconeLocalRepro
{
    internal class Program
    {
        static async Task Main(string[] args)
        {
            // v0.7.0 works with 2.1 client
            const string Image = "ghcr.io/pinecone-io/pinecone-local:v0.7.0";
            const ushort FirstPort = 5080;
            const int IndexServiceCount = 10;

            ContainerBuilder builder = new ContainerBuilder()
                .WithImage(Image)
                // Pinecone Local will run on port $FirstPort.
                .WithPortBinding(FirstPort, assignRandomHostPort: true)
                // We are currently using the default Pinecone port (5080), but we can change it to a random port.
                // In such case, we are going to need to set the PORT environment variable to the new port.
                .WithEnvironment("PORT", FirstPort.ToString());

            for (int indexService = 1; indexService <= IndexServiceCount; indexService++)
            {
                // And the index services on the following ports.
                builder = builder.WithPortBinding(FirstPort + indexService, assignRandomHostPort: true);
            }

            var container = builder.Build();

            await container.StartAsync();

            Dictionary<int, int> containerToHostPort = Enumerable.Range(FirstPort, IndexServiceCount + 1)
                .ToDictionary(port => port, port => (int)container.GetMappedPublicPort(port));

            UriBuilder baseAddress = new()
            {
                Scheme = "http",
                Host = container.Hostname,
                Port = container.GetMappedPublicPort(FirstPort)
            };

            ClientOptions clientOptions = new()
            {
                BaseUrl = baseAddress.Uri.ToString(),
                MaxRetries = 0,
                IsTlsEnabled = false,
                GrpcOptions = new()
                {
                    HttpClient = new(new RedirectHandler(containerToHostPort), disposeHandler: true)
                    {
                        BaseAddress = baseAddress.Uri
                    },
                }
            };

            PineconeClient pc = new(
                apiKey: "ForPineconeLocalTheApiKeysAreIgnored",
                clientOptions: clientOptions);

            try
            {
                var indexName = "example-index3";
                await pc.CreateIndexAsync(new CreateIndexRequest
                {
                    Name = indexName,
                    Dimension = 2,
                    Metric = CreateIndexRequestMetric.Cosine,
                    Spec = new ServerlessIndexSpec
                    {
                        Serverless = new ServerlessSpec
                        {
                            Cloud = ServerlessSpecCloud.Aws,
                            Region = "us-east-1",
                        }
                    },
                    DeletionProtection = DeletionProtection.Disabled,
                });

                Console.WriteLine("Index created successfully");

                var index = pc.Index(indexName);
                // Upsert records into the index
                var req = new UpsertRequest()
                {
                    //Namespace = "example-namespace",
                    Vectors = new List<Vector>
                    {
                        new Vector
                        {
                            Id = "vec1",
                            Values = new ReadOnlyMemory<float>([1.0f, -2.5f]),
                            Metadata = new Metadata {
                                ["genre"] = new("drama"),
                            },
                        },
                        new Vector
                        {
                            Id = "vec2",
                            Values = new ReadOnlyMemory<float>([3.0f, -2.0f]),
                            Metadata = new Metadata {
                                ["genre"] = new("documentary"),
                            },
                        },
                        new Vector
                        {
                            Id = "vec3",
                            Values = new ReadOnlyMemory<float>([0.5f, -1.5f]),
                            Metadata = new Metadata {
                                ["genre"] = new("documentary"),
                            }
                        }
                    }
                };

                await index.UpsertAsync(req);
            }
            finally
            {
                await container.StopAsync();
            }
        }
    }

    internal sealed class RedirectHandler : DelegatingHandler
    {
        private readonly Dictionary<int, int> _containerToHostPort;

        public RedirectHandler(Dictionary<int, int> portRedirections)
            : base(new HttpClientHandler())
        {
            _containerToHostPort = portRedirections;
        }

        protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
        {
            // When "host" argument is not provided for PineconeClient.Index,
            // it will try to get the host from the Pinecone service.
            // In the cloud environment it's fine, but with the local emulator
            // it reports the address with the container port, not the host port.
            if (request.RequestUri != null && request.RequestUri.IsAbsoluteUri
                && request.RequestUri.Host == "localhost"
                && _containerToHostPort.TryGetValue(request.RequestUri.Port, out int hostPort))
            {
                UriBuilder builder = new(request.RequestUri)
                {
                    Port = hostPort
                };
                request.RequestUri = builder.Uri;
            }

            return base.SendAsync(request, cancellationToken);
        }
    }
}

[eddumelendez]

First of all, the emulator seems to be outdated and does not work with 3.0 of the client (the client reports that some required JSON fields are missing), but we can get it working with 2.1.

@adamsitnik had the same problem with 0.0.7, can you try v1.0.0.rc0?

[jseldess]

@eddumelendez, @adamsitnik, unfortunately, pinecone local does not support collections. There are a list of unsupported features in the docs here: https://docs.pinecone.io/guides/operations/local-development#limitations. I'm not sure if that's the entire problem, but it's likely part of it.

[adamsitnik]

@adamsitnik had the same problem with 0.0.7, can you try v1.0.0.rc0?

Thanks @eddumelendez ! With v1.0.0.rc0 I was able to get it working with the 3.0 client for .NET 8.

For the Full .NET Framework I am getting some GRPC 13 errors (INTERNAL). After attaching a debugger it seems that HTTP 2.0 is enforced, which is not supported and the communication fails.