Markdown streaming puts each token in a separate paragraph

[nat-n]

I'm hacking together a custom response renderer where I want to be able to have some logic to edit the stream as it is rendered but still benefit from streaming markdown rendering.

He's my attempt to create a component that could allow me to do this:

const CustomResponseRenderer: ResponseRenderer<string> = (props) => {
  let markdownRef = useRef(null)
  let tokenCount = useRef(0)

  let mdStreamParser: MarkdownStreamParser

  if (markdownRef.current && props.content.length) {
    mdStreamParser = createMarkdownStreamParser(markdownRef.current, {
      waitTimeBeforeStreamCompletion: 2000,
    })
  }

  useEffect(() => {
    mdStreamParser?.next(props.content.slice(tokenCount.current).join(''))
    tokenCount.current = props.content.length
  }, [props.content.join('')])

  return (
    <div>
      <div ref={markdownRef} />Y
    </div>
  )
}

The weird this is that what gets rendered into the referenced div is a <p> paragraph per token, that is per call to mdStreamParser?.next. There are no line breaks in the text. Am I doing something wrong?