To use LaTeX in Conversations, enable MathJax for the course in Site Info. See How do I add LaTeX language to my course site?
LaTeX may be used inline in both posts (questions or discussions) and answers within the Conversations tool. Type the LaTeX content into the appropriate text field. Upon saving the post, the content will display with LaTeX formatting.