There are 20 Memoir Pages in total (which the quest giver mentions at some point). The final one (progression-wise) can be found in the Tower of Babil.
Regarding the crafting side quest, though it has already been mentioned elsewhere on this forum: Adamantium Ore is at least a confirmed drop (not steal) from Black Dragons in the Temple of Kaison (other rare monsters there might drop it too, or not). Vamammoth Fang is a drop (not steal) from Vamammoth, which can be encountered in the ice region on the world map outside of the Tower of Babil. Speaking of which: Pure Nethicite, if I remember correctly, can be found in a treasure chest inside the Tower of Babil.
Bring Edea's Tome to Altney Archives. Then Oalston can learn one of the two ultimate black magic spells. Please refer to Overworld Map 3 in the game's "maps and guides" folder if you have trouble reaching Altney Archives (it's located on the southeastern continent).
I also only managed to obtain 6 Mystic Keys (5 from slaying the Elemental Fiends to gain access to the Elemental Abyss, which is mandatory, and 1 for defeating Bahamut and the eight successor dragons), so I don't know if or how all weapons in the Armory of the Ancients can be claimed.