Schmidtt's research(pdf) appears to present a comparatively continuous drop from the time of European Speak to until eventually about the beginning with the twentieth century (at which position their population started expanding once again). The one bit in there that really sticks out like