How do I know if my memories are literal or symbolic?

This comes down to belief. If you can tolerate believing the memory is real, it's real. If you can't, you can still symbolically interpret and gain value from it.