Nicotinamide adenine dinucleotide (NAD) is a central metabolite in cellular processes. Depletion of NAD has been demonstrated to be a prevalent theme in both prokaryotic and eukaryotic immune responses. Short prokaryotic Argonaute proteins (Agos) are associated with NADase domain-containing proteins (TIR-APAZ or SIR2-APAZ) encoded in the same operon. They confer immunity against mobile genetic elements, such as bacteriophages and plasmids, by inducing NAD depletion upon recognition of target nucleic acids. However, the molecular mechanisms underlying the activation of such prokaryotic NADase/Ago immune systems remain unknown. Here, we report multiple cryo-EM structures of NADase/Ago complexes from two distinct systems (TIR-APAZ/Ago and SIR2-APAZ/Ago). Target DNA binding triggers tetramerization of the TIR-APAZ/Ago complex by a cooperative self-assembly mechanism, while the heterodimeric SIR2-APAZ/Ago complex does not assemble into higher-order oligomers upon target DNA binding. However, the NADase activities of these two systems are unleashed via a similar closed-to-open transition of the catalytic pocket, albeit by different mechanisms. Furthermore, a functionally conserved sensor loop is employed to inspect the guide RNA-target DNA base pairing and facilitate the conformational rearrangement of Ago proteins required for the activation of these two systems. Overall, our study reveals the mechanistic diversity and similarity of Ago protein-associated NADase systems in prokaryotic immune response.