Conversations Reloaded

[noplanman]

I've been putting a lot of thought into redesigning the conversations and how we could make it easier for users to create totally customised conversations.

Here's what I've come up with so far. It's completely up for discussion. Let's collect ideas and suggestions here to come up with a great solution! 😃
The class names are open to be changed to something better if anyone has any ideas.

Also note, this is more like a "what i would like to be able to do as a user", as opposed to a "this is simple to implement". So there is no real code for this just yet, meaning that we'd need to figure out how best to implement the final solution.

I hope you understand my current solution as it's present in my mind. Just ask if something is unclear and I'll update this issue.

Here are a list of required classes so far:

ConversationItemType class

An enum of different entity types. It would make sense to not have this class but instead have a general abstract class as an enum for all entity types. What I mean is, have a class that can be used anywhere in the library (for example EntityType), to compare for entity types. So instead of giving the message a $type string, it would get a specific const of an enum, e.g. EntityType::DOCUMENT.

e.g.

abstract class ConversationItemType // or rather "EntityType"
{
  const TEXT     = 1;
  const AUDIO    = 2;
  const DOCUMENT = 4;
  const PHOTO    = 8;
  const STICKER  = 16;
  const VIDEO    = 32;
  const VOICE    = 64;
  const CONTACT  = 128;
  const LOCATION = 256;
  const VENUE    = 512;
  const ANY      = 1023; // Next bit - 1
}

ConversationItem class

Each part of the conversation is an object of this class.

It's made up of 2 core methods:

• The initial text that gets outputted when the item is called
• The process that handles the reply and decides what data should be saved for the current item

And a bunch of extra ones that help control the conversation (see below)

To enforce a clean splitting up of the code, all necessary parts are pointers to callback functions.

Callback functions would be for the initial output, the handling of the reply, the custom "invalid type" text, the custom check if a valid reply has been made by the user, etc. (in case there are any others)

Each item has a unique id, so that the state of the conversation can be remembered and the input of each item can be saved properly.

Additionally, we assign the next item to an already existing item, thus making a tree of items. This "tree" is the flow of the conversation.

e.g. Create an item that asks for the name and another one that asks for the age. We then assign the one asking for the age, to the one asking for the name. This way, the conversation class knows which item comes next.

Expanding on this, would be the ability to define multiple next items, depending on the input by the user.

e.g. Conversation item says: "Send a photo or document"
We define the next conversation item if a photo is sent, and a different one if a document is sent.

Each item also has an array of flags that tell the item which reply types are accepted

e.g. When asking the user for a document, the "Document" flag would be set, letting the conversation item know if the correct input has been made by the user.

If not, a custom "wrong input format" output could be defined and the initial text is outputted to the user.

Conversation class

This class is the one that manages the whole conversation, interacting with the database, fetching and storing data etc.
(similar to the way it is now)

The top-most conversation item gets added to the conversation object to start the conversation.

Additional necessary changes

Add a default __toString() method to all entities, that would return a default string that makes sense.

e.g. $location->__toString() would return: Latitude: <lat> - Longitude: <lng>

A simple pseudo-code example could be (inside a command):

<?php

function execute()
{
    $conversation = Conversation::load($user_id, $chat_id, ...);

    $accepted_types_1 = ConversationItemTypes::DOCUMENT | ConversationItemTypes::PHOTO;
    $root_conversation_item = new ConversationItem(
        'doc_or_photo', // Unique ID.
        [&$this, 'init_callback_1'], // Either a proper callback function...
        [&$this, 'response_callback_1'], // Either a proper callback function...
        $accepted_types_1
    );
    $root_conversation_item->setInvalidTypeResponse([&$this, 'invalid_type_response_callback_1']); // Either a proper callback function...

    $accepted_types_2 = ConversationItemTypes::LOCATION;
    $location_conversation_item = new ConversationItem(
        'loc',
        'Please specify a location', // ...or just text.
        [&$this, 'response_callback_2'],
        $accepted_types_2
    );
    $location_conversation_item->setInvalidTypeResponse('Only a location object allowed...'); // ...or just text.

    $anything_conversation_item = new ConversationItem(
        'anything',
        'Please send anything, really...',
        'Anything received, no idea what though :-/', // ...or just text. If null, default __toString() of $response_message (would need to be defined first!)
        ConversationItemTypes::ANY
    );

    // Add next items.
    // If a photo is passed, next item is $location_conversation_item...
    $root_conversation_item->addNextItem($location_conversation_item, ConversationItemType::PHOTO);
    // else $anything_conversation_item.
    $root_conversation_item->addNextItem($anything_conversation_item, ConversationItemType::ANY);

    // Add the root conversation item.
    $conversation->addRootItem($root_conversation_item);

    // Start the conversation. This checks which state we're in to skip ahead to the proper place if necessary.
    $conversation->start();
}


function init_callback_1($conversation, $conversation_item)
{
    return 'Please upload a document or photo';
}
function response_callback_1($conversation, $conversation_item, $response_message)
{
    $doc   = $response_message->getDocument();
    $photo = $response_message->getPhoto();
    $data  = null;
    if ($doc) {
        $data = $doc->getFileId();
    } elseif ($photo) {
        $data = end($photo)->getId();
    }
    return $data;
}
function invalid_type_response_callback_1($conversation, $conversation_item, $response_message)
{
    $response = 'Either a document or photo please, nothing else.';
    if ($response_message->getLocation()) {
        $response .= "\n" . 'Especially NOT a Location!';
    }
    return $response;
}

function response_callback_2($conversation, $conversation_item, $response_message)
{
    $loc = $response_message->getLocation();
    return 'Lat: ' . $loc->getLatitude() . ' - Lng: ' . $loc->getLongitude();
}

[noplanman]

@juananpe @akalongman @MBoretto @jacklul

May the discussion regarding the new conversation structure begin!

[juananpe]

Hi there,

Good job @noplanman ! I'm still digging into your proposal, but some questions and recommendations popped up already.

• First, in the pseudo-code, I think that there is a typo here:
  // If a photo is passed, next item is $location_conversation_item...
  $root_conversation_item->addNextItem($location_conversation_item, ConversationItemType::PHOTO);
  If I understood correctly, the last parameter should be ConversationItemType::LOCATION instead of PHOTO, shouldn't it?
• And the comment in that same excerpt of code should point out that "If a photo or a document is passed", to be consistent with the example.
• In ConversationItem class, I'm wondering if the ID should be a private attribute instead of allowing the developer to define it (like 'doc_or_photo', 'loc',etc.). Are we going to have to manipulate that reference or is just something internal for indexing the state of the conversation? If so, I would propose to generate a private uniqueid() internally.
• I have been studying how this Conversation pattern has been programmed in other frameworks&languages. For example, in BotKit [https://github.com/howdyai/botkit#multi-message-replies-to-incoming-messages] there is a queue of Conversation Items. After been consumed, an item has to call conversation.next() as its last instruction to trigger the next item of the queue. I like the BotKit's PizzaTime multi-stage conversation example, I think that it clearly explains how to implement this behavior: https://github.com/howdyai/botkit#multi-stage-conversations
  It also allows to naturally integrate some cancel-like commands in a conversation (they call them Conversation Control Functions https://github.com/howdyai/botkit#conversation-control-functions , like sayFirst, stop, repeat, silentRepeat and the aforementioned next).

We could borrow some ideas from there :)

• Finally, I wonder if a Conversation state could be implemented internally as a finite state machine using an already existing library like Finite PHP (or any other similar one): https://github.com/yohang/Finite . At the end of the day, I see a conversation as "a Stateful object by defining states and transitions between these states" (the very same definition of what Finite PHP does !)

Just my two cents!

[noplanman]

@juananpe Thanks for the great feedback!

First, in the pseudo-code, I think that there is a typo here:
// If a photo is passed, next item is $location_conversation_item...
$root_conversation_item->addNextItem($location_conversation_item, ConversationItemType::PHOTO);
If I understood correctly, the last parameter should be ConversationItemType::LOCATION instead of PHOTO, shouldn't it?

The second parameter for the addNextItem() method is meant to define which the next item is based on the type received from the user. So for the given example, the first command allows either a DOCUMENT of PHOTO to be sent. If a PHOTO is sent, then the next item should be $location_conversation_item. If ANY other is sent (in this case it can only be DOCUMENT), call the $anything_conversation_item.
The idea behind this, is to fine tune the flow of the conversation. It may seem a bit complex at first, if you have a better idea for handling this I'm all ears!

Is it more clear now?

And the comment in that same excerpt of code should point out that "If a photo or a document is passed", to be consistent with the example.

If my explanation above is clear, the comment should make sense now.

In ConversationItem class, I'm wondering if the ID should be a private attribute instead of allowing the developer to define it (like 'doc_or_photo', 'loc',etc.). Are we going to have to manipulate that reference or is just something internal for indexing the state of the conversation? If so, I would propose to generate a private uniqueid() internally.

The reason why I allow the user to define it, is so that the value of any conversation item response can then easily be fetched. e.g. $conversation->getItemData('loc'); or whatever. If this is an automatically generated ID, the user wouldn't be able to fetch the data that easily, as the ID wouldn't be known.

I have been studying how this Conversation pattern has been programmed in other frameworks&languages. For example, in BotKit [https://github.com/howdyai/botkit#multi-message-replies-to-incoming-messages] there is a queue of Conversation Items. After been consumed, an item has to call conversation.next() as its last instruction to trigger the next item of the queue. I like the BotKit's PizzaTime multi-stage conversation example, I think that it clearly explains how to implement this behavior: https://github.com/howdyai/botkit#multi-stage-conversations
It also allows to naturally integrate some cancel-like commands in a conversation (they call them Conversation Control Functions https://github.com/howdyai/botkit#conversation-control-functions , like sayFirst, stop, repeat, silentRepeat and the aforementioned next).

I really like this approach and have thought of this as well. It's nice and easy. I was trying to focus on something that requires less coding by the user for complex conversations. Using the above example with the Document and Photo replies, checking which one was passed would need to be done in the ask function itself (going by the example of the JS library).
But now that you've shown me this example, we'll definitely take it into account!

We could borrow some ideas from there :)

Yes!!! The beauty of open source 😊

Finally, I wonder if a Conversation state could be implemented internally as a finite state machine using an already existing library like Finite PHP (or any other similar one): https://github.com/yohang/Finite . At the end of the day, I see a conversation as "a Stateful object by defining states and transitions between these states" (the very same definition of what Finite PHP does !)

Right, this would change the entire way of looking at conversations, yes? Using this, each conversation item would be a state and each step of the conversation would be a transition. Is this what you imagined?

Could be interesting to explore!

Just my two cents!

Valuable two cents, thanks! 😃 Keep 'em coming...

[juananpe]

The second parameter for the addNextItem() method is meant to define which the next item is based on the type received from the user

OK, that makes sense. Much clearer now. And the comment to that section of code also applies (and yep, I was wrong :)

The reason why I allow the user to define it, is so that the value of any conversation item response can then easily be fetched. e.g. $conversation->getItemData('loc');

Now I get it and it also confirms that the design is consistent. Great!

[BotKit...] I was trying to focus on something that requires less coding by the user for complex conversations

I buy your argument here! :)

[Finite library...] Right, this would change the entire way of looking at conversations, yes? Using this, each conversation item would be a state and each step of the conversation would be a transition. Is this what you imagined?

Yes, following your second example, we would have item1 as the initial state of the conversation. Then, we could go (a transition) to item2 (final state) if the user uploads a photo or to item3 if it was a document, etc. As you pointed out, I think that introducing this Finite library would indeed suppose a deep change to the current conversation design but I wanted to discuss it before taking any decision that could make this integration harder to implement in the future :-P

Keep up the good work!

[noplanman]

I think that introducing this Finite library would indeed suppose a deep change to the current conversation design but I wanted to discuss it before taking any decision that could make this integration harder to implement in the future :-P

Which is absolutely perfect, that's why we're discussing this before going crazy with an implementation 👍

[MBoretto]

Finally I join the conversation conversation!
@noplanman @juananpe Thanks for the great job!
I like the overall proposal you made!

Some thoughts for names:
ConversationItem -> ConversationStage
ConversationItemType -> ConversationInputType

I like the idea of callbacks, would be nice if possible support also anonymous function (lamda).
Will the finite library suit completely our needs? If yes we have to exploit it!

Is not clear to me how can be possible to jump a state for example:
A->B->C (state of my conversation)
I would like to have to possibility to jump from A to C as a function of the response of A.

I'm also wonder if can i branch the conversation something like:
A---B--C
|
D--E
In this case i will change the conversation branch as a function of the response of A.

Seems to me a problem that can be solved with some graph approach.

Let me know if i can be more clear!

[noplanman]

Finally I join the conversation conversation!

👍 😄

ConversationItem -> ConversationStage
ConversationItemType -> ConversationInputType

👍 I like your suggestions! Makes it more clear.

I like the idea of callbacks, would be nice if possible support also anonymous function (lamda).

Absolutely! The callback is basically a "function", be it a function object (anonymous function) or a pointer to a function (call_user_func([$obj, 'func']))

Will the finite library suit completely our needs? If yes we have to exploit it!

First need to check this and see how it could be used. But I agree, if it does provide all the "framework" functionality we'd need, it could be a great solution.

Is not clear to me how can be possible to jump a state for example:
A->B->C (state of my conversation)
I would like to have to possibility to jump from A to C as a function of the response of A.
I'm also wonder if can i branch the conversation something like:
A---B--C
|
D--E
In this case i will change the conversation branch as a function of the response of A.

Take a look at the second diagram I posted above, the one with the fork. It would be possible to set up rules for a more complex structure. This does need some more thought put into it though, to fully support what you mean. I could imagine a callback function that handles that.
In the response_callback_1 function for example, we could do the more complex checks, like to define the next conversation step depending on the number input from the last step, etc. Having access to the $conversation parameter, we could then choose the next conversation step and jump into it using a ->execute() (or whichever name) method to call it directly, skipping the predefined execution order.

The examples I made are simpler, with routes that will always be the same, depending on only simple factors, namely response type. This offers the users an easy way to build semi-complex conversations really easily, with no callback function confusion. But as you say, the ability to fully customise the whole conversation is a must, we just need to find a nice solution together 😃

Seems to me a problem that can be solved with some graph approach.

I agree! It's like the examples mentioned on the finite basic graph page.