Courses taught schedule

[ajkiessl]

Looking for some ideas on how to get the Courses Taught data imported consistently on the 4th or 5th week of the semester. While this sounds simple enough, it is actual a bit of a challenge. We get the data from LionPATH, which starts giving us courses for a semester after the 4th week of the semester. Ideally, we'd like to run the import within a week of that. We don't really have a great way of knowing when the semester starts; it changes every year. We also can't just check the SFTP server for a new file, since it gives us a new file — but not necessarily new data — every week. We could do some kind of checksum, but there may be small changes that happen weekly that we don't want to import. Or we could just find better dates to run these on.

Right now the process runs on the first of February, May, July, August, October, and December.

Also, we want the import to run again at the end of the semester. To import any changes that happen throughout the semester.

One thing I've considered in the past, and am not sure if it would be okay to do, would be to just run this import every other week like the OSP data import.

[nmg110]

What I would like to see, is we pull the data from the SFTP server every month either on the 1st or the 25th. Then we do a data check with what data is in AI with what is in the file, if there is a change update the data in AI, if there is no record in AI then import the data, and lastly if that course is no longer in the file for that user we remove the course from AI.
That would give us accurate and up to date data through out the semester.

[ajkiessl]

If we are okay with it running once a month, could we bump the frequency up to every other week? That would guarantee we never have a new classes import beyond the 5th week after the semester starts.

if that course is no longer in the file for that user we remove the course from AI

This will likely be tricky, but could potentially be done. We could run a comparison against last weeks file and find what's been remove, but then we wouldn't know the IDs of those records in AI to delete them. Or we could pull the data from AI and compare (giving us the IDs in AI), but that could be a slow process and consume a lot of resources. This will require some planning/thinking.

[nmg110]

I think running it every other week and every month may be too much, because it doesn't change that often, but we need to get it in sooner than we are now. Fall is when most of the faculty (2nd, 4th, and 6th year) reviews are being done, which is why we need the data in sooner for this semester.
Now that I can use the FAMS Tool to do the import for me, maybe I just setup a reminder to pull the data the 2nd week of classes and then the automation can continue with any updated information later.