Hello Ztakeo and thank you for reporting this issue !
Did you pick the potion visible behind the charlatan's stand in the courtyard ? Once you picked up the potion, you can speak to him.
What do you mean by "it won't let me talk to him anymore" ?
I am also encountering this problem. I went to his cart killed the thugs and picked up the potion and 2 notes. However upon returning to him the option to talk to him does not appear. I did go to his room at the inn and gather the evidence that proves he is an alchemist before returning to him however so perhaps this has caused the bug. I am playing on PS4 if that helps at all.
I just started a new game and tried the quest again, this time not going to his room at the inn before returning to him and can confirm that the bug did not happen this time. Seems it's going to his room and getting the info first that causes the problem. hope this has been helpful in fixing the issue for a future patch.