Troubleshooting GitHub Sync Issues: Missing Files & Folders | Dust Community

Troubleshooting GitHub Sync Issues: Missing Files & Folders

Any ideas why GitHub would not sync all files (after selecting “Code Synchronization = Yes”)? I’ve noticed few files & folders missing, no idea why 😕

33 comments

Alban
·
·
Hi, I've started a full resync of your github connection, give it a few hours and let us know if things look better!
👍1
Gregor
·
·
Alban it already says “Last Sync ~3 mins ago”, that means its done and I can check?
Alban
·
·
Let me check if the worker is still running
Alban
·
·
yep looks complete
Gregor
·
·
files & folders still missing
Gregor
·
·
for example there should be a folder “patches” in the root but its not showing up
Gregor
·
·
Also, I’m not seeing PRs, only “Issues” (which we don’t use so its understandably empty). Not even Issues are showing up now
Gregor
·
·
Alban any updates on the above?
Alban
·
·
👀
Alban
·
·
what are the file extensions in the patches folder?
Gregor
·
·
Alban*.patch
Alban
·
·
ok that's an extension we need to whitelist
Alban
·
·
give me a sec
Alban
·
·
https://github.com/dust-tt/dust/pull/11602 will go live in the next deploy 🙂
Gregor
·
·
Alban what about missing PRs section? I only see the files. I used to see empty Issues (since we don’t use them)
Gregor
·
·
I see bunch of suffixes missing that would explain other files missing:
.csv .neon .phtml .twig .xhtml .xml .xsd
Can you add those into the whiltelist also?
Gregor
·
·
Alban any feedback on the above?
Alban
·
·
Hi, We're unfortunately not going to add csv and xml files as this can generate a imports with sensitive data on a lot of customers. I'll see to add the others!
Alban
·
·
I don't think we're syncing PRs
Gregor
·
·
Alban for CSV i understand but XML are in some cases used for data-structure and its good for AI to be able to look into it (for example - Magento ecommerce platform uses XML extensively) . Maybe having a configuration in the Admin UI to list whats whitelisted and allow to expand (this way its not OOTB but gives customers control of the data). As for PRs => https://docs.dust.tt/docs/github-connection it says:
Dust integrates with Github to bring your repository’s Issues, Pull Requests, Discussions, and codebase- if enabled
Thats why I asked 🙂 Pull Requests do have alot of value since they give the insight on how things are fixed/changed which should be valuable information.
Gregor
·
·
Alban any ETA on new release to have the new whitelisted files?
Gregor
·
·
Alban any updates on this? The repo is still missing alot of files 😞
Alban
·
·
Hello hello, sorry not monitoring the community slack daily
Alban
·
·
I'll start a full resync of your github
Gregor
·
·
Alban is the updated whitelist released?
Gregor
·
·
oh, i see them in repo 😍
Alban
·
·
Yep
Alban
·
·
I've restarted your sync
Alban
·
·
So should be popping up 🙂
Alban
·
·
Sorry about the delay - for any "real" support it's better to send an email to support@dust.tt - this is a community slack so we're monitoring it lighlty
Gregor
·
·
Alban regarding XMLs - maybe you can support additional whitelisting from the repository itself? Something like .dust.yaml that would define whitelist for a project/repo?
Alban
·
·
That would require some work on our side, and we'll have to wait for other customers to want to ingest XML files to consider it 👍
Gregor
·
·
ok understood. I wish you would just include it into the whitelist 😇 😄 But I’ll wait