When scholars refer to the “Synoptic Problem,” they mean the literary relationship between Matthew, Mark and Luke (the first three canonical gospels can be arranged and compared in a synopsis while the Gospel of John is very different from the “Synoptic” tradition). Indeed, there must be some kind of literary relationship since the Synoptic Gospels often agree in order, wording and even down to parenthetical asides such as “let the reader understand” (Mk 13:14; Matt 24:14). The three most common solutions to this tricky puzzle are the Griesbach hypothesis (Matthew and Luke wrote first, Mark a later abridgement of the two), the Two-Documentary Hypothesis (Mark wrote first and was independently used by Matthew and Luke, while the shared material in Matthew and Luke goes back to a single Greek document labelled “Q” [from German Quelle or source]) and the Farrer or Goulder-Goodacre hypothesis (Mark wrote first, but dispenses with “Q” as Luke used Matthew). There are also more chaotic approaches to the Synoptic problem that may allow for Luke’s knowledge of Matthew and argues that material in the double tradition (the so-called “Q” material shared by Matthew/Luke) may go back to a variety of Greek and Aramaic sources. For more in depth treatment, see Stephen Carlson’s excellent site on the Synoptic Problem including a number of diagrams of potential solutions and an annotated bibliography. Or Mark Goodacre, who made his introductory textbook The Synoptic Problem: A Way Through the Maze available online, has many useful podcasts on this question.
I am not going to get into the whole Synoptic Problem but focus on Mark. While Augustine’s solution that Mark was a mere follower and abbreviator of Matthew (De Consensu Evangelistarum 1.3) held sway for approximately 1400 years, I am in agreement with the huge majority of NT scholars that Mark was the first gospel written and the source for Matthew and Luke (i.e. Markan priority). I find convincing the arguments from order (Mark tends to be the middle term, so Matthew and Luke rarely agree in wording or order against Mark), length (Mark is the shortest of the three and almost entirely reproduced in Matthew and Luke), writing style (Matthew/Luke often edit and clean up Mark’s awkward grammar or style, omit Aramaic words, etc), and harder readings (Matthew/Luke seem to correct Mark’s christology, eschatology, ambivalent view of disciples, etc). Moreover, Mark would make a rather odd summary of Matthew or abridgement of Matthew and Luke: why would Mark cut out the birth narratives, much of the ethical teaching (e.g. Sermon on the Mount or Plains) and the resurrection appearances (assuming the Gospel ends at 16:8), or why would Mark choose to add details to individual pericopes shared in the triple tradition (stories shared by all three Synoptics) such as that Jesus’ family had thought he had gone out of his mind (Mark 3.19-21), that Jesus’ initial attempt to heal the blind man was not completely successful (‘I can see people, but they look like trees, walking.’) (Mark 8:22-25), that the fig tree Jesus cursed for not having any fruit because “it was not the season for figs” (Mark 11:14) and so on (it makes much more sense that Matthew and Luke omitted these potentially embarrassing passages)? Here are some examples from a handout I used when teaching a class on this (taken from the RSV) in which I asked students in each example to tell me which passage they think was earlier and why:
And he did not do many mighty works there, because of their unbelief (Matthew 13.58)
And he could do no mighty work there, except that he laid his hands upon a few sick people and healed them. And he marvelled because of their unbelief. (Mark 6.5-6)
“Save, Lord; we are perishing!” (Matt 8.25)
“Teacher, do you not care if we perish?” (Mark 4.38)
“Master, Master, we are perishing!” (Luke 8.24)
‘Truly I say to you, there are some standing here who will not taste death before they see the Son of Man coming in his kingdom.’ (Matthew 16.28)
And he said to them, ‘Truly I tell you, there are some standing here who will not taste death before they see that the kingdom of God has come with power.’ (Mark 9.1)
Then Jesus was led up by the Spirit into the wilderness… (Matthew 4:1; cf. Luke 4:1)
The Spirit immediately drove [from ekballō] him out into the wilderness (Mark 1:12)
“Why do you ask me about what is good? One there is who is good.” (Matt 19:17)
“Why do you call me good? No one is good but God alone.” (Mark 10:18; cf. Luke 18:19)
…and behold a leper came to him and knelt before him, saying, ‘Lord, if you will, you can make me clean.’ And he stretched out his hand and touched him, saying, ‘I will; be clean!’ (Matthew 8.2-3)
And a leper came to him beseeching him, and kneeling said to him, ‘If you will, you can make me clean.’ Moved with anger [textual variant: compassion; see here for more discussion], Jesus stretched out his hand and touched him, and said to him, ‘I will; be clean!’ (Mark 1.40-42)
…when he saw Jesus, he fell on his face and besought him, ‘Lord, if you will, you can make me clean.’ And he stretched out his hand and touched him, and said, ‘I will; be clean.’ (Luke 5.12-13).