examples files

[LYDE1234]

my yaml file references an external file with examples:

  parameters:
    - name: 'id'
      in: 'path'
      description: 'Id of the object'
      required: true
      schema:
        type: 'string'
        format: 'guid'
      examples:
        camera: 
          $ref: 'examples.json'

but when I run my raft_local.py job, I get this compiler error:

Unhandled exception. System.AggregateException: One or more errors occurred. (Could not resolve the JSON path '/work_dir_7bd549a8-96e8-4301-8b02-0e3d2125e37b/examples.json' with the full JSON path '/work_dir_7bd549a8-96e8-4301-8b02-0e3d2125e37b/examples.json'.)
---> System.InvalidOperationException: Could not resolve the JSON path '/work_dir_7bd549a8-96e8-4301-8b02-0e3d2125e37b/examples.json' with the full JSON path '/work_dir_7bd549a8-96e8-4301-8b02-0e3d2125e37b/examples.json'.
---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
---> System.IO.FileNotFoundException: Could not find file '/work_dir_7bd549a8-96e8-4301-8b02-0e3d2125e37b/examples.json'.

clearly RESTler in the docker container tries to find the examples file in its workdir but can't. How do you get RAFT to copy the examples files into the container so that RESTler can resolve them?

[stishkin]

You can pass the folder with examples to docker container by mounting it. See this example:

rest-api-fuzz-testing/cli/samples/restler/running-against-raft-common-file-share/fuzz.json

Line 4 in b992047

"readOnlyFileShareMounts": [

[LYDE1234]

that example shows how to pass compile output as input to later RESTler stages. How do you tell the compiler to use the file share mount that you set up in the config? If I only mount a share as shown in the example, I still get the error.

[stishkin]

you can mount any file share. The example passes a result of compile step to run.

But you can create an example share same way and pass it to compile step.

see this example:
https://github.com/microsoft/rest-api-fuzz-testing/blob/main/cli/samples/restler/replay-a-bug/replay-common-file-share.json

this example mounts a file share to replay a RESTler bug.

So you can mount any file share and pass it to RESTler

[LYDE1234]

I have this configuration:

{
"readOnlyFileShareMounts": [
{
"fileShareName": "specifications",
"mountPath": "/specifications"
},
{
"fileShareName": "examples",
"mountPath": "/examples"
}
],...
"tasks": [
{
"toolName": "RESTler",
"outputFolder": "restler",
"keyVaultSecrets": [ "TOKEN" ],
"authenticationMethod": {
"Token": "Token"
},
"targetConfiguration": {
"apiSpecifications": [
"http://host.docker.internal:8000/openapi_cams_only.yaml"
],
"endpoint": "http://host.docker.internal/managementserver/rest"
},
"toolConfiguration": {
"tasks": [
{
"task": "Compile",
"runConfiguration": {
"inputFolderPath": "/examples"
},...

but I still get the same error - does the swagger compile step know to look in the share mount or do I need to do something else in my configuration?

[stishkin]

you need to change your Compile task and ExamplesDirectory there:

        /// The directory where the compiler should look for examples.
        /// If 'discoverExamples' is true, this directory will contain the
        /// example files that have been discovered.
        /// If 'discoverExamples' is false, every time an example is used in the
        /// Swagger file, RESTler will first look for it in this directory.
        ExamplesDirectory : string

"tasks": [
  {
    "toolName": "RESTler",
    "outputFolder": "compile",
    "toolConfiguration": {
      "task": "Compile",
    "compileConfiguration" : {
     "exampleConfigFilePath " : "/examples"
    }
    }
  }
]

Here is a good example that uses compile configuration section 👍
https://github.com/microsoft/rest-api-fuzz-testing/blob/main/cli/samples/restler/running-against-raft/compile.json

[LYDE1234]

I added this to my configuration:

"compileConfiguration":
{
"useRefreshableToken": true,
"discoverExamples": false,
"useBodyExamples": true,
"useQueryExamples": true,
"examplesDirectory": "Examples",
"exampleConfigFilePath": "/examples"
}

and I changed the reference in my yaml file to:

examples:
camera:
$ref: '../examples/examples.json'

now I get a different error so i assume that worked:

Unhandled exception. System.AggregateException: One or more errors occurred. (Unable to cast object of type 'NJsonSchema.JsonSchema' to type 'NSwag.OpenApiExample'.)
---> System.InvalidCastException: Unable to cast object of type 'NJsonSchema.JsonSchema' to type 'NSwag.OpenApiExample'.
at NJsonSchema.References.JsonReferenceBase1.NJsonSchema.References.IJsonReferenceBase.set_Reference(IJsonReference value) at NJsonSchema.JsonSchemaReferenceUtilities.JsonReferenceUpdater.VisitJsonReferenceAsync(IJsonReference reference, String path, String typeNameHint) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action`1 replacer)

my examples.json file looks like this:

{
"parameters": {
"id": "09b11ea3-085c-4641-a84a-ce2f68a09a53"
}
}

[stishkin]

Where do you get the failure from ? is it happening in RAFT or RESTler ?

[LYDE1234]

It's coming from RESTler- here's the full stack trace:

Unhandled exception. System.AggregateException: One or more errors occurred. (Unable to cast object of type 'NJsonSchema.JsonSchema' to type 'NSwag.OpenApiExample'.)
---> System.InvalidCastException: Unable to cast object of type 'NJsonSchema.JsonSchema' to type 'NSwag.OpenApiExample'.
at NJsonSchema.References.JsonReferenceBase1.NJsonSchema.References.IJsonReferenceBase.set_Reference(IJsonReference value) at NJsonSchema.JsonSchemaReferenceUtilities.JsonReferenceUpdater.VisitJsonReferenceAsync(IJsonReference reference, String path, String typeNameHint) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj) at NJsonSchema.JsonSchemaReferenceUtilities.JsonReferenceUpdater.VisitAsync(Object obj) at NJsonSchema.JsonSchemaReferenceUtilities.UpdateSchemaReferencesAsync(Object rootObject, JsonReferenceResolver referenceResolver, IContractResolver contractResolver) at NJsonSchema.Infrastructure.JsonSchemaSerialization.FromJsonAsync[T](String json, SchemaType schemaType, String documentPath, Func2 referenceResolverFactory, IContractResolver contractResolver)
at NSwag.OpenApiDocument.FromJsonAsync(String data, String documentPath, SchemaType expectedSchemaType, Func2 referenceResolverFactory) at NSwag.OpenApiDocument.FromFileAsync(String filePath) --- End of inner exception stack trace --- at Microsoft.FSharp.Control.AsyncResult1.Commit() in F:\workspace_work\1\s\src\fsharp\FSharp.Core\async.fs:line 349
at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronouslyInCurrentThread[a](CancellationToken cancellationToken, FSharpAsync1 computation) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\async.fs:line 882 at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronously[T](CancellationToken cancellationToken, FSharpAsync1 computation, FSharpOption1 timeout) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\async.fs:line 890 at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T](FSharpAsync1 computation, FSharpOption1 timeout, FSharpOption1 cancellationToken) in F:\workspace_work\1\s\src\fsharp\FSharp.Core\async.fs:line 1154
at Restler.Workflow.swaggerDocs@88-1.Invoke(String fp) in D:\a\1\s\src\compiler\Restler.Compiler\Workflow.fs:line 88
at Microsoft.FSharp.Primitives.Basics.List.map[T,TResult](FSharpFunc2 mapping, FSharpList1 x) in F:\workspace_work\1\s\src\fsharp\FSharp.Core\local.fs:line 248
at Microsoft.FSharp.Collections.ListModule.Map[T,TResult](FSharpFunc2 mapping, FSharpList1 list) in F:\workspace_work\1\s\src\fsharp\FSharp.Core\list.fs:line 75
at Restler.Workflow.generateGrammarFromSwagger(String grammarOutputDirectoryPath, FSharpOption1 swaggerDoc, Config config) in D:\a\1\s\src\compiler\Restler.Compiler\Workflow.fs:line 88 at Restler.Workflow.generateRestlerGrammar(FSharpOption1 swaggerDoc, Config config) in D:\a\1\s\src\compiler\Restler.Compiler\Workflow.fs:line 247
at Program.main(String[] argv) in D:\a\1\s\src\compiler\Restler.CompilerExe\Program.fs:line 37

[stishkin]

@marina-p
Does this crash look like an obvious fix in user configs ? Or RESTler update required ?

[marina-p]

@LYDE1234 If you are referencing the example from the specification, and they are in the same directory (or a sub-directory with relative path from the specification), you do not need the ExampleConfigFilePath or an example config file. If that works for you, just omit the above file path from config.json, put the example in the same directory as the spec, and hopefully the issue will be resolved.

I believe the reason for your error is you are referencing the example config file in your specification yaml. The specification should only reference actual examples (like the one you provided which contains "id"). The example config file should only be referenced in config.json, and the example config file contains a list of request types, which point to one or more examples.

See https://github.com/microsoft/restler-fuzzer/blob/main/docs/user-guide/Examples.md for more details.