Regional police chief, Olivier de Mazieres, said the attack took place at Saint-Charles station at 1:45 pm (1145 GMT) on Sunday.
“Two victims have been stabbed to death,” De Mazieres said without elaborating.
According to later reports, the victims were two women, one of whom was fatally stabbed in stomach while the other had her throat slit by the assailant.
Xavier Tarabeux, the local prosecutor, said that policemen at the scene of the attack had shot dead the assailant.
Police in Marseille urged people to avoid Saint-Charles station. There was no comment on the motives of the attacker although a source close to the investigation, speaking on condition of anonymity, said the man had carried out the assault in a way similar to the operatives of Daesh, a Takfiri group that has killed dozens of people in attacks across France since early 2015.
French Interior Minister Gerard Collomb said on his Twitter page that he would immediately travel to the Mediterranean port city to examine the situation.
Later on Sunday, French anti-terror prosecutors said they had opened an investigation into the knife attack in Marseille.
The prosecutor's office in Paris was quoted by the AFP as saying in a statement that the probe would focus on "killings linked to a terrorist organization" and the "attempted killing of a public official," both of which are terror-related charges.
A French police source has said that the man suspected of carrying out the knife attack in Marseille was known to authorities for common law crimes, said a French police source.
French President Emmanuel Macron expressed his outrage at the Marseille attack on his Twitter account, saying, "I hail the Opération Sentinelle soldiers and the police forces who reacted with extreme calmness and efficiency."
France is still on high alert as sporadic attacks happen across the country by Daesh operatives or people expressing loyalty to the terrorist group. Some 7,000 troops have been deployed in cities and towns as part of the Opération Sentinelle to guard high-risk areas such as transport hubs, tourist sites and religious buildings.
A total of 239 people, excluding the two victims of the Sunday attack in Marseille, have been killed in attacks in France since January 2015. The deadliest of all the attacks came in November that year when terrorists brutalized the capital city of Paris, killing more than 120.