can I work on this?

can I work on this?

No objections from me 🙂

Just hit this as well. I think the right solution would be to allow trailing commas. It's a bit more obvious with TOML and shouldn't really differ in ini, UX-wise.

I haven't checked, but my hunch is that configparser might be reading this entire value as a giant string, like file1,file2,file3,, splitting it by comma. And then, the last element is an empty string. This translates to just calling MyPy in terminal and passing an empty string on CLI.

I have confirmed that passing that empty string as another file breaks in exactly the same way:

$ MYPY_PATH=src/ mypy --explicit-package-bases myproject/package1 myproject/package2 myproject/module1.py myproject/module2.py ''

Additionally, I've found that you can call it with a double empty string:

$ MYPY_PATH=src/ mypy --explicit-package-bases '' ''

and it would explode the same...

Interestingly, a single empty string argument seems fine:

$ MYPY_PATH=src/ mypy --explicit-package-bases ''

So it seems to me that there are 2 bugs here:

1. MyPy's argument processing breaks when there's more than one file path argument passed and at least one of those paths is an empty string — this deserves a clear error output, regardless of whether it's coming from a config or a CLI arg
2. The configparser-based reader should normalize the INI format to allow for trailing commas and disallow empty strings so that it behaves the same as TOML from the user perspective.

@hauntsaninja could you please change the label from documentation to bug? Also, this is probably a good-first-issue.

It just occurred to me that the config parsing bug may affect other settings that are being interpreted as lists. Perhaps, that's worth checking by whoever gets to work on this.

Hi!, Can I take this up?

@hauntsaninja thanks for the labels! Could you assign the issue to me and @x612skm? I'll help him navigate the process.

@x612skm @webknjaz Feel free to work on this! (we generally don't assign issues)

You can check out CONTRIBUTING.md and the Developer Guides for contributing guidelines and tips on where to start. Thanks!

Just for the fyi - I've started brainstorming into this and soon will be hitting a PR after a discussion with @webknjaz . Thanks!

@brianschubert I've got some feedback for you.

We've found a weakness in the contributing docs. I asked Soubhik to write some tests and since it was a new module, the first idea was to start with something simple, like a test function. It took some time to notice that it wasn't being picked up by pytest. This was when I asked to push the change w/o a fix to the PR (#18472) to make sure it's failing, and it didn't.

I was able to find the respective pytest config that disables the default test collection, but I don't believe that a less experienced person would know where to look. Still, it was quite frustrating to troubleshoot.

I checked the contrib and other docs — there seems to be no mention of needing to use classes in test modules.

So the action item is to document this somewhere, the closer to the section on invoking the tests the better.

Hi @x612skm @webknjaz I looked into this issue with all my attention and I have found why this issue exists. Can I create a pr for this if it's okay with you guys. This is my first pr related to bug so I wanted to do this. Anyway the issue has to do with how empty path is processed in mypy function create_source_list

    for path in paths:
        path = os.path.normpath(path)
        if path.endswith(PY_EXTENSIONS):
            # Can raise InvalidSourceList if a directory doesn't have a valid module name.
            name, base_dir = finder.crawl_up(path)
            sources.append(BuildSource(path, name, None, base_dir))
        elif fscache.isdir(path):
            sub_sources = finder.find_sources_in_dir(path)
            if not sub_sources and not allow_empty_dir:
                raise InvalidSourceList(f"There are no .py[i] files in directory '{path}'")
            sources.extend(sub_sources)
        else:
            mod = os.path.basename(path) if options.scripts_are_modules else None
            sources.append(BuildSource(path, mod, None))
    return sources

what happens is when we have trailing comma or if we pass empty string in cmd line we get empty string as input. Then path = os.path.normpath(path) this normalises it to '.' so this '.' acts as root dir, therefore, we will get dublicates of previously processed dirs as well ultimately. @amin-nejad although it is almost clear but the reason why you found it to read first element again has to do with how mypy deals with this situation. Once it finds it's first duplicate it doesn't further proceed checking.

[mypy] files = myproject/package1, myproject/package2, myproject/module1.py, myproject/module2.py,

Below you can see how it perfectly processes 4 sources and on from 5th which would be an empty string '' it starts to repeat again since it treats this as root dir.

[BuildSource(path='myproject/package1/__init__.py', module='package1', has_text=False, base_dir='/Users/prabhatthapa/PycharmProjects/mypy_practice/myproject', followed=False), 

BuildSource(path='myproject/package2/__init__.py', module='package2', has_text=False, base_dir='/Users/prabhatthapa/PycharmProjects/mypy_practice/myproject', followed=False), 

BuildSource(path='myproject/module1.py', module='module1', has_text=False, base_dir='/Users/prabhatthapa/PycharmProjects/mypy_practice/myproject', followed=False), 

BuildSource(path='myproject/module2.py', module='module2', has_text=False, base_dir='/Users/prabhatthapa/PycharmProjects/mypy_practice/myproject', followed=False), 

BuildSource(path='./myproject/package1/__init__.py', module='package1', has_text=False, base_dir='/Users/prabhatthapa/PycharmProjects/mypy_practice/myproject', followed=False), 

BuildSource(path='./myproject/package2/__init__.py', module='package2', has_text=False, base_dir='/Users/prabhatthapa/PycharmProjects/mypy_practice/myproject', followed=False), 

BuildSource(path='./myproject/module1.py', module='module1', has_text=False, base_dir='/Users/prabhatthapa/PycharmProjects/mypy_practice/myproject', followed=False), 

BuildSource(path='./myproject/module2.py', module='module2', has_text=False, base_dir='/Users/prabhatthapa/PycharmProjects/mypy_practice/myproject', followed=False), 

BuildSource(path='./venv/bin/activate_this.py', module='activate_this', has_text=False, base_dir='/Users/prabhatthapa/PycharmProjects/mypy_practice/venv/bin', followed=False), BuildSource(path='./main.py', module='main', has_text=False, base_dir='/Users/prabhatthapa/PycharmProjects/mypy_practice', followed=False)]

@webknjaz I have a suggestion if we just add

if not path:
     continue

this will deal with the errors perfectly from both issues related to empty strings from command lines and config file since it makes no sense to normalise empty string path. Although we can deal with trailing comma issues separately at split_and_match_files_list(paths.split(",")). But if we need to have error message for empty strings we can do that as well.

@Prabhat-Thapa45 yes, the issue (two actually) has been identified with the reproduces known and Soubhik even started a draft PR. I think, he's stuck on writing tests for this. If you want to collaborate, maybe start by offering the tests and send a PR against the branch in his fork.

Yeah I can help him on that. @webknjaz I assumed that we want to have better error message as well from this MyPy's argument processing breaks when there's more than one file path argument passed and at least one of those paths is an empty string — this deserves a clear error output, regardless of whether it's coming from a config or a CLI arg so will Soubhik work on this? so far I saw his changes will only work for ini files only if it comes to empty string from cmd line this will still throw the similar error.