Skip to content

Conversation

@SHiLLySiT
Copy link

This aims to implement #53, which IMO, has some huge benefits:

  1. There is no longer separate source .INK files and compiled .JSON files that can get out of sync. Which means designers don't need to remember commit both .INK and JSON files, which can happen if they're testing/writing outside Unity with Inky.
  2. No need to deal with managing a ink file compilation queue, and fighting with the Editor. Ink files are compiled when imported, either when added or changed.
  3. You can expose InkFile references instead of generic TextAsset references in game code.
  4. This codebase becomes significantly less complex, and less prone to weird errors that have to deal with compiling assets outside of Unity's normal import process.

This PR is currently in a proof of concept stage: it hasn't been fully tested, but loading and running ink files at runtime and in the Editor is working.

The original implementation was written before Unity supported custom scripted importers, so many of the systems are now obsolete. Since this is a pretty big change, I wanted to open a PR to get an initial review to get some initial thoughts from the maintainers.

Interested in all feedback, but the question at the top of my mind: Do we need to expose includes and master files in the editor? While Ink at a language level still has these concepts, I'm not sure how important it is to expose these in Unity if its no longer important for compilation?

@SHiLLySiT SHiLLySiT marked this pull request as draft September 2, 2024 02:12
@tomkail
Copy link
Collaborator

tomkail commented Sep 2, 2024

Oh wow thanks for looking into this! I'll take a look this week if I find time. At a glance it looks lovely (so much less code!!)
I've pinged the Unity dev community on Discord (https://discord.gg/inkle) to take a look too, since this is a big change and I'd like as many devs as possible to confirm that it doesn't affect them negatively in any way.

@SHiLLySiT
Copy link
Author

Totally! I see your message in #ink-unity-integration-dev. I'll keep watch for questions/comments there too. Thanks!

@SHiLLySiT SHiLLySiT marked this pull request as ready for review October 6, 2024 23:45
@SHiLLySiT
Copy link
Author

@tomkail Okay I'm moving this out of draft as I've cleaned up the known things I needed to do, but I suspect there may be things I've missed since this was such a huge change. Feel free to tag me on any changes you want after doing a review. I'll loop back when I have the time 🤘

@tomkail
Copy link
Collaborator

tomkail commented Oct 7, 2024 via email

@SHiLLySiT
Copy link
Author

SHiLLySiT commented Oct 8, 2024

Sounds good to me! I'd rather give more people more time to test it out before its merged into main 😅

I don't appear to have the option to change the base branch to a new (non-existing) branch tho. If you make a beta branch I can update the PR!

@SHiLLySiT
Copy link
Author

I forgot to update the docs, so I snuck another commit in 👍

Comment on lines -114 to +115
DefaultAsset asset = AssetDatabase.LoadAssetAtPath<DefaultAsset>(path);
DrawInkFile(InkLibrary.GetInkFileWithFile(asset), rect);
InkFile asset = AssetDatabase.LoadAssetAtPath<InkFile>(path);
DrawInkFile(asset, rect);
Copy link

@ErnSur ErnSur Nov 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

InkFile is now a Scriptable Object.
No OnDrawProjectWindowItem would be needed if we assign a custom icon to InkFile.cs from the inspector

@SHiLLySiT
Copy link
Author

@ErnSur

I left some general comments, moving to scripted importers is a natural step forward for the library but it looks like it will require breaking changes.

Hey, thank you for the review! I just wanted to drop a note here that I probably won't have time to review and make changes until around Feb 2025.

This could be a chance for some cleanups and other breaking changes around the codebase. Let me know if you want more feedback, I'm interested in closer collaboration.

While I'm in favor of cleaning up the code, I'm not a maintainer and this would probably be a better conversation to have with @tomkail. But I'll certainly circle back for a re-review once I've had a chance to process your feedback.

Cheers!

@tomkail
Copy link
Collaborator

tomkail commented Nov 25, 2024 via email

@SHiLLySiT
Copy link
Author

any big breaking changes should be tested on large scale projects before merging to the main branch (which is really the only reason this hasn’t been merged in, it’s fantastic stuff)

@tomkail I've been advocating for using Ink in our projects at work, so when the next one spins up, I'll use my fork with these changes to get them tested.

@SHiLLySiT
Copy link
Author

@tomkail @ErnSur I think I've stumbled on to one of reasons why the import process prior to this PR had been so complex - the Ink language's INCLUDE feature.

The Ink language supports including other Ink files, which allows them to be broken up into smaller chunks. I had overlooked this initially, so to support this I added a bit of code in 0b5980d (which ErnSur had also reccomended in this comment) so that master ink files were aware of their includes. This solved the problem of Ink files used as includes not triggering reimports of their master files.

However the problem I now have is that some Ink files used as includes cannot be compiled on their own outside of the context of the master file that includes them. For example, when a master file defines global variables, any include files that use the global variables fail to independently compile. This is problem for the changes introduced in this PR because the custom importer will flag such files as failed to compile, when they really should just be ignored.

And this is where I'm stuck.

I'm not quite sure if Unity has a way to express this kind of dependency with the ScriptedImporter API. While its possible to know if a master file has includes (we can parse it to look for INCLUDE keywords) there's no way to know if an Ink file is an include file. Do either of you (maybe more directed at @ErnSur) have a suggestion on how to handle this?

I think I'd like to avoid a system similar to the old InkLibrary but I feel like we may not be able to get around generating some sort of intermediary file to allow for a two-pass import?

@SHiLLySiT
Copy link
Author

Ok here's are two proposals, both centered around not trying to auto identify if files are used as includes:

  1. We allow Ink files to fail to compile. We also switch logging of compilation errors to warnings instead of errors. I think seeing red error messages in Unity's console indicate something you must fix for the game to work properly. In this case, an Ink script that fails to compile is intended behavior. We'd still keep the error icon on Ink assets so developers can easily identify which ones have errors.
  2. Use a ScriptedImporterEditor to introduce a SkipCompilation flag. This would be similar to the "Should also be Master File" flag, except in reverse. All files would be treated as a master file and compiled, but specific files can be opted out with SkipCompilation enabled.

@ErnSur
Copy link

ErnSur commented Mar 25, 2025

However the problem I now have is that some Ink files used as includes cannot be compiled on their own outside of the context of the master file that includes them.

the custom importer will flag such files as failed to compile, when they really should just be ignored.

there's no way to know if an Ink file is an include file

Some API context:

  • Any ScriptedImporter can declare serializable fields that can be modified in the inspector and act as importer settings for the file.
  • ScriptedImporter can create more than one asset from a file. One asset needs to be main asset and other the rest of them are so-called sub-assets. In editor it appears as scriptable object with a foldout that hides other scriptable objects.

So my idea for a solution to this issue would be to import ink variables as additional scriptable object (sub-asset). Then InkImporter could have a public InkScriptVariables[] ScriptVariables field which can be optionally populated.

  • When InkImporter is importing a script it will try to "compile" in the context of ScriptVariables variables if any are referenced.
  • When InkImporter is importing a script that has include files it will try to find them in the project and add its variables asset reference to their ScriptVariables collection.

I don't know how feasible is the second point, but I'd try this approach. I think it should be possible to load another InkImporter from OnImport callback.

Use a ScriptedImporterEditor to introduce a SkipCompilation flag. This would be similar to the "Should also be Master File" flag, except in reverse. All files would be treated as a master file and compiled, but specific files can be opted out with SkipCompilation enabled.

As I mentioned before, you don't really have to introduce any custom editors to add serializable state to the importer. Just add a serializable filed to importer the same way you would add it to a scriptable object and it'll work.

@SHiLLySiT
Copy link
Author

Thanks for your input!

Any ScriptedImporter can declare serializable fields that can be modified in the inspector and act as importer settings for the file.
As I mentioned before, you don't really have to introduce any custom editors to add serializable state to the importer. Just add a serializable filed to importer the same way you would add it to a scriptable object and it'll work.

Oops! I hadn't realized I could just defined them in the importer, thanks for calling this out 👍

So my idea for a solution to this issue would be to import ink variables as additional scriptable object (sub-asset). Then InkImporter could have a public InkScriptVariables[] ScriptVariables field which can be optionally populated.

  • When InkImporter is importing a script it will try to "compile" in the context of ScriptVariables variables if any are referenced.
  • When InkImporter is importing a script that has include files it will try to find them in the project and add its variables asset reference to their ScriptVariables collection.

I don't know how feasible is the second point, but I'd try this approach. I think it should be possible to load another InkImporter from OnImport callback.

Ohh interesting. I'm not sure how the separate ink variables will solve the core problem? As your second bullet is where the issue is, specifically determining a file is an include before the file that actually includes it is imported.

@ErnSur
Copy link

ErnSur commented Apr 1, 2025

Seems I didn't explain the idea that well...

specifically determining a file is an include before the file that actually includes it is imported.

importer wouldn't have to successfully import ink file if its missing variable declarations.

Let's say we have a project with following files:

  • File A - main file
    • Contains a reference (include) to File B
    • Defines global variable X
  • File B - companion file
    • Requires variable X definition

Let's say we import the project and assets are imported in unfavorable order for us where File B is being being imported before file A:

  • File B import: fails to import the asset since it cannot find variable X definition
  • File A import:
    • Generates the InkVariables object that contains the variable X definition
    • Goes over all ink files included in the main file and re-imports them with the newly created InkVariables object. With that they should be able to be imported correctly.

Importer design issue

Even if the above solution fixes the issue I would actually change the way importer works.
Ink Importer should not fail to import the file if its missing variable definitions.

We're trying to import File B but its missing variable X? No problem, it should report importer warning but still succeed at file import. The result will have "{variable name}" in places where the variable should be.
The moment you populate the InkVariables array with a reference to the InkVariables asset (the idea I described here) the file will reimport.

As a source of inspiration I recommend looking at Unity UXML and USS files, how they're imported and how they deal with references between each other.

@SHiLLySiT
Copy link
Author

Ahh thanks for the explanation!

I'm still not sure what the additional InkVariable objects get us, but I think the second half of your last post hits on what I've been trying to describe:

Importer design issue
Even if the above solution fixes the issue I would actually change the way importer works.
Ink Importer should not fail to import the file if its missing variable definitions.

We're trying to import File B but its missing variable X? No problem, it should report importer warning but still succeed at file import. The result will have "{variable name}" in places where the variable should be.
The moment you populate the InkVariables array with a reference to the InkVariables asset (the idea I described #205 (comment)) the file will reimport.

As a source of inspiration I recommend looking at Unity UXML and USS files, how they're imported and how they deal with references between each other.

If I'm understanding correctly, I think we're on the same page here. Specifically, if an Ink file fails to compile (and by compile I mean compile via the Ink compiler) it should still be successfully imported into Unity.

However, what I'm still unsure about is the developer experience for this. Going with your example, if File B is imported first and logs a warning regarding the compilation failure to the Unity console, how should we follow up once File A is imported? Additionally, importing File A still doesn't fix the compilation of File B because File B is not intended to be compiled as a main file.

I'm starting to feel more strongly for proposal #2 from my earlier comment as I think this is likely the same issue the original developers faced and why the master file toggle was added.

@tomkail Do you have any historical context you could share?

…s that aren't intended to be compiled standalone
@ARPP3
Copy link

ARPP3 commented Aug 27, 2025

Hey all! What is the status of this PR?

Reading through the thread, it seems like this was slowed down by the INCLUDE statements. Testing it out locally, it seems super promising. I also was running into frustrations with the sidekick files and seeing the single asset feels really like a great step for this package. I actually only came across this fork after trying to do the same myself 😅 Would be a shame for this to become outdated,

@SHiLLySiT @tomkail @ErnSur

@tomkail
Copy link
Collaborator

tomkail commented Aug 27, 2025 via email

@SHiLLySiT
Copy link
Author

SHiLLySiT commented Aug 27, 2025

We've been using my fork at work for two projects, with one of them being quite sizeable. I had been waiting to confirm until it shipped in a few months, but at this point the team has been using it for over a year so I'd say it's pretty stable.

I think the main impact is people with existing projects that want to upgrade. It's going to probably require a lot of manual changes for them to migrate to this new version.

@tomkail
Copy link
Collaborator

tomkail commented Aug 27, 2025 via email

@SHiLLySiT
Copy link
Author

@tomkail I wouldn't recommend putting this behind a feature toggle. We'd need to restore all the now deleted code and refactor the refactoring I did to support both systems 😅 I'm also not quite sure what Unity would do with all the imported Ink scripts when the custom importer is disabled.

I'd recommend a major version bump and include a migration guide. I can write the guide - would you prefer it as a Markdown file in the repo, a comment in this PR, or somewhere else?

@reallyfancy
Copy link

Just catching up on this now. It looks really promising!

I have some thoughts on the outstanding question about the best workflow for INCLUDE dependencies.

Unless I'm misunderstanding something, I don't think that exposing global variables as sub-assets would solve all of the problems of missing INCLUDE files - for example, INCLUDE files can also contain function definitions.

To my mind, thoroughly capturing, parsing, and resolving all these dependencies is outside the scope of the original intended change - and very welcome improvement! - here.

I'd suggest that the pragmatic option is for the user to continue to explicitly identify which files are master files. As well as being a familiar concept and workflow for existing users, this would greatly reduce the scope of work required to land this update, and therefore its potential for bugs.

If, in the future, there's a desire to build on this further and add some kind of automatic dependency resolution, that can be treated as a separate PR.

@SHiLLySiT
Copy link
Author

@reallyfancy Hey thanks for jumping in here!

I have some thoughts on the outstanding question about the best workflow for INCLUDE dependencies.

Unless I'm misunderstanding something, I don't think that exposing global variables as sub-assets would solve all of the problems of missing INCLUDE files - for example, INCLUDE files can also contain function definitions.

To my mind, thoroughly capturing, parsing, and resolving all these dependencies is outside the scope of the original intended change - and very welcome improvement! - here.

Yup! This is also where I landed, so we're in agreement there.

I'd suggest that the pragmatic option is for the user to continue to explicitly identify which files are master files. As well as being a familiar concept and workflow for existing users, this would greatly reduce the scope of work required to land this update, and therefore its potential for bugs.

If, in the future, there's a desire to build on this further and add some kind of automatic dependency resolution, that can be treated as a separate PR.

Ahh that is a great argument for keeping the old terminology. Its been awhile now so I don't quite remember my reasoning for the name change 🤔 I just updated the PR to rename the new toggle!

@SHiLLySiT
Copy link
Author

@tomkail I also added some migration notes to the changelog to communicate changes to users.

@reallyfancy
Copy link

Thanks for getting back to me!

With regard to the naming, there's generally a move away from calling things "master" in favour of maintaining/root/whatever makes contextual sense, and I'm definitely not opposed to that - but the exact name that feels like a decision for maintainers!

The important consideration from my perspective is limiting change for users - in this case, letting them keep thinking in terms of opting in to standalone compilation, rather than opting out.

Anyway, this is looking good! I'm happy to take a look at this, likely in a week or so, and experimentally port a medium-sized Ink project to it to see what happens.

@tomkail I see that you expressed concerns about this change not being behind a feature flag, but @SHiLLySiT doesn't think that's the right fit for this code. It would be good to figure out what would give you the confidence to merge this. It's a big change but there are a lot of ways to derisk things like this - additional safety, additional testing, major version bump, migration tooling, and so on. I'm happy to pitch in to help this land, if we can come up with a plan!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants